Search

Abstract This letter proposes a low‐power keyword spotting (KWS) architecture based on a modified temporal efficient neural network (TENet) and a simplified mel‐frequency cepstrum coefficient (MFCC) algorithm. The optimized MFCC algorithm reduces the computational load by 82% for multiplications and 66% for additions. An efficient hardware architecture and data flow for TENet have been designed, resulting in a 3.1× reduction in the operating cycle compared to similar work. The parameter count and computational load are reduced by 3.7× and 2.8×, respectively, and the accuracy reaches 95.36% for ten keywords in the Google Speech Command Dataset (GSCD). Operating at a frequency of 16 KHz for MFCC and 100 KHz for NN accelerator on a 28 nm process, the power consumption overhead is 9.1 µW.

Abstract The target of text‐based person re‐identification (Re‐ID) is to retrieve the corresponding image of a person through the given text information. However, due to the homogeneous variety and modality heterogeneity, it is challenging to simultaneously learn both global‐level and local‐level cross‐modal features and align them in the same embedding space without additional networks. To address these problems, an effective multi‐level cross‐modality learning (MCL) framework for language and vision person Re‐ID is proposed. More specifically, a multi‐branch feature extraction (MFE) module is designed to comprehensively map both global and partial semantic information for the visual and textual embedding at the same time, capturing the intra‐class semantic relationships in multi‐granularities. Besides, a cross‐modal alignment (CA) module is devised to match the multi‐grained representations and reduce the inter‐class gap from global‐level to partial‐level. Extensive experiments conducted on the CUHK‐PEDES and ICFG‐PEDES datasets suggest that this method outperforms the state‐of‐the‐art models.

Abstract Breathing condition is an essential physiological indicator closely related to human health. Wearable flexible breath sensors for respiration pattern recognition have attracted much attention as they can provide physiological signal details for personal medical diagnosis, health monitoring, etc. However, present smart mask based on flexible breath sensors using single-mode detection can only detect a relatively small number of respiration patterns, especially lacking the ability to accurately distinguish mouth breath from nasal one. Herein, a smart face mask incorporated with a dual-sensing mode breathing sensor that can recognize up to eight human respiration patterns is fabricated. The breathing sensor uses novel three dimensional (3D) buckling carbon nanofiber mats as active materials to realize the function of pressure and temperature sensing simultaneously. The pressure model of the sensors shows a high sensitivity that are able to precisely detect pressure generated by respiratory airflow, while the temperature model can realize non-contact temperature variation caused by breath. Benefit from the capacity of real-time recognition and accurate distinguishing between mouth breath and nasal breath, the face mask is further developed to monitor the development of mouth breathing syndrome. The dual-sensing mode sensor has great potential applications in health monitoring.

Abstract Image‐to‐image translation is a class of vision and graphics problems where the goal is to learn the mapping between input images and output images. However, due to the unstable training and limited training samples, many existing GAN‐based works have difficulty in producing photo‐realistic images. Herein, dual‐directional generative adversarial networks are proposed, which consist of four adversarial networks, to produce images of high perceptual quality. In this framework, self‐reconstruction strategy is used to construct auxiliary sub‐networks, which impose more effective constraints on encoder‐generator pairs. Using this idea, this model can increase the use ratio of paired data conditioned on the same dataset and obtain well‐trained encoder‐generator pairs with the help of the proposed cross‐network skip connections. Moreover, the proposed framework not only produces realistic images but also addresses the problem where condition GAN produces sharp images containing many small, hallucinated objects. Training on multiple supervised datasets, convincing evidences are shown to prove that this model can achieve compelling results by latently learning a common feature representation. Qualitative and quantitative comparisons against other methods, demonstrate the effectiveness and superiority of the method.

Abstract RGB‐Infrared person re‐identification (RGB‐IR Re‐ID) is a task aiming to retrieve and match person images between RGB images and IR images. Since most surveillance cameras capture RGB images during the day and IR images at night, RGB‐IR Re‐ID is helpful when checking day and night surveillance for criminal investigations. Previous related work often only extracts sharable and identity‐related features in images for identification. Few researches specifically extract and make use of features that do not have the ability to distinguish identity, e.g. identity‐unrelated features derived from background and modality. In this Letter, we propose a novel and concise RGB‐IR Re‐ID network named two‐way constraint network (TWCN). Compared with traditional Re‐ID networks, TWCN not only extracts and utilises identity‐related features but also novelly makes full use of identity‐unrelated features to improve the accuracy of the experiment. TWCN uses a reverse‐triplet loss to extract identity‐unrelated features, and proposes an orthogonal constraint to remove identity‐unrelated information from identity‐related features, which improves the purity of identity‐related features. In addition, a correlation coefficient synergy and central clustering (CCSCC) loss is introduced into TWCN to extract identity‐related features effectively. Extensive experiments have been conducted to prove our method is effective.

In this paper, we introduce a low-cost, expansible, and compatible measurement and control system for atomic force microscopes (AFM) based on a quartz tuning fork (QTF) self-sensing probe and frequency modulation, which is mainly composed of an embedded control system and a probe system. The embedded control system is based on a dual-core OMAPL138 microprocessor (DSP + ARM) equipped with 16 channels of a 16-bit high-precision general analog-to-digital converter (ADC) and a 16-bit high-precision general digital-to-analog converter (DAC), six channels of an analog-to-digital converter with a second-order anti-aliasing filter, four channels of a direct digital frequency synthesizer (DDS), a digital input and output (DIO) interface, and other peripherals. The uniqueness of the system hardware lies in the design of a high-precision and low-noise digital—analog hybrid lock-in amplifier (LIA), which is used to detect and track the frequency and phase of the QTF probe response signal. In terms of the system software, a software difference frequency detection method based on a digital signal processor (DSP) is implemented to detect the frequency change caused by the force gradient between the tip and the sample, and the relative error of frequency measurement is less than 3%. For the probe system, a self-sensing probe controller, including an automatic gain control (AGC) self-excitation circuit, is designed for a homemade balanced QTF self-sensing probe with a high quality factor (Q value) in an atmospheric environment. We measured the quality factor (Q value) of the balanced QTF self-sensing probes with different lengths of tungsten tips and successfully realized AFM topography imaging with a tungsten-tip QTF probe 3 mm in length. The results show that the QTF-based self-sensing probe and the developed AFM measurement and control system can obtain high quality surface topography scanning images in an atmospheric environment.

Recently, great advances had been made by using scanning probe microscopy (SPM) to quantify the relative permittivity of thin film materials on a nanometer scale. The imaging techniques of permittivity for thin film materials with SPM, especially for photoelectric materials, have not been fully researched until now. Here, we presented a method to image permittivity of thin film materials by using a scanning capacitance microscope (SCM). This method combined the quantitative measurement by using SCM with the capacitance gradient–distance fitting curve to obtain the two-dimensional (2D) permittivity image at room temperature under atmospheric conditions. For the demonstration, a 2D permittivity image of film of molybdenum oxide (MoO3), a kind of photoelectric material, was acquired. From the image, it could be found that the average values of permittivity of MoO3 film and of MoO3 film-doped NaCl were about 8.0 and 9.5, respectively. The experimental results were quantitatively consistent with other experimental results of the same material. The reported technique here could provide a novel method for imaging the relative permittivity with nanometer resolution and be helpful for the study of photoelectric materials.

Recent work in unsupervised image-to-image translation by adversarially learning mapping between different domains, which cannot distinguish the foreground and background. The existing methods of image-to-image translation mainly transfer the global image across the source and target domains. However, it is evident that not all regions of images should be transferred because forcefully transferring the unnecessary part leads to some unrealistic translations. In this paper, we present a positional attention bi-flow generative network, focusing our translation model on an interesting region or object in the image. We assume that the image representation can be decomposed into three parts: image-content, image-style, and image-position features. We apply an encoder to extract these features and bi-flow generator with attention module to achieve the translation task in an end-to-end manner. To realize the object-level translation, we adopt the image-position features to label the common interesting region between the source and target domains. We analyze the proposed framework and provide qualitative and quantitative comparisons. The extensive experiments validate that our proposed model is qualified to accomplish the object-level translation and obtain compelling results with other state-of-the-art approaches.

In this paper, we propose a novel image inpainting framework that takes advantage of holistic and structure information of the broken input image. Different from the existing models that complete the broken pictures using the holistic features of the input, our method adopts Patch-generative adversarial networks (GANs) equipped with multi-scale discriminators and edge process function to extract holistic, structured features, and restore the damaged images. After pre-training our Patch-GANs, the proposed network encourages our generator to find the best encoding of the broken input images in the latent space using a combination of a reconstruction loss, an edge loss, and global and local guidance losses. Besides, the reconstruction and the global guidance losses ensure the pixel reliability of the generated images, and the remaining losses guarantee the contents consistency between the local and global parts. The qualitative and quantitative experiments on multiple public datasets show that our approach has the ability to produce more realistic images compared with some existing methods, demonstrating the effectiveness and superiority of our method.

Recent advances in deep learning, especially deep convolutional neural networks, have led to great performance improvement over semantic segmentation systems. Unfortunately, training deep neural networks (DNNs) requires a humongous amount of labeled data, which is laborious and costly to collect and annotate. Thus, plenty of works have proposed an alternative solution to ease the training set creation by using synthetic data. However, models trained on these kinds of data usually under-perform on real images due to the well-known issue of domain shift. To address it, we propose a generative adversarial network (GAN)-based framework called category-level adversarial adaptation networks (CAA-Nets) for domain adaptation in the context of semantic segmentation. Considering semantic predictions that contain spatial and structure information of images, our idea is to make use of this character by imposing discriminators on the semantic predictions. Different from existing works, the proposed framework utilizes a category-level discriminator in the output space to shrink the gap between real and synthetic images. Similar to reinforcement learning, we take final results as a guide to update parameters in the right direction by use of the output-based discriminator. Moreover, to further enhance the performance, we construct an image-based generator and discriminator pair to distill the feature representations obtained by a DNN. Taking advantage of these modules, our model can achieve competitive performance compared with some existing methods. To showcase the generality and scalability of our approach, we evaluate the proposed method on the Cityscapes dataset by adapting from GTAV and SYNTHIA datasets, where the results demonstrate the effectiveness of our method.

Temporal action detection, as a branch of video analysis, aims to locate the time points when the actions start and end, and classify the actions occurred in videos into correct categories. Generating high-quality proposals is a key step in temporal action detection task. In this paper, we introduce a novel network, named multi-scale proposal regression network (MPRN), for temporal action proposal generation. First, we take encoding visual features as input and predict action scores for time points, in order to group them to generate rough proposals. Then, we regress the proposal's boundaries to obtain more precise proposals via our multi-scale proposal regression network. Compared with SSN and TURN, our multi-scale regression segments are characterized by flexible boundaries. Experiments show that 1) Our method is better than other proposal generation methods on THUMOS-14 dataset and ActivityNet-v1.3 dataset. 2) The effectiveness of our method is due to its own architecture, not the selection of visual feature encoders. 3) Our proposal generation method can generate temporal proposals for unseen action classes, which shows the good generalization ability of our proposal generation method.

In this paper, we propose a Resetting-Label Network based on Fast Group Loss for Person Re-Identification (RLFGL-ReID). The major challenge of Re-ID lies in how to preserve the similarity of the same person against large variations caused by complex background, different illuminations, and various view angles while discriminating different individuals. To address the above-mentioned problems, we propose the RLFGL-ReID that includes resetting-label (RL) and fast group loss (FGL). Two main contributions of our network are as follows. First, a new method, the resetting-label method, which resets the ID labels, is proposed for the Re-ID network. Resetting the ID labels of each pedestrian is beneficial in maximizing the inter-group distances between each people, achieving better performance on classifying different individuals. Second, a fast group loss, i.e., an advanced version of variance group loss (VGL), is proposed to simplify the training process and accelerate the loss computation. By doing so, the network can eliminate the restriction of inputting the whole group of data when training the network. To confirm the effectiveness of our method, we extensively conduct our method on several widely used person Re-ID benchmark datasets. As the result shows, our method achieves rank@1 accuracy of 98.38% on CUHK03, 95.46% on Market1501, and 91.2% on DukeMTMC-reID, outperforming the state-of-the-art methods and confirming the advantage of our method.

Early compressor trees based on carry-save adders and single-column parallel counters show good performance in ASIC design, but do not adapt well to modern field-programmable gate arrays (FPGAs). Recently, compressor trees built from generalized parallel counters (GPCs) were synthesized on FPGAs to address this issue. Despite the improved timing performance of GPC-based compressor trees, area reduction is not as significant as delay, and can be further optimized. In this paper, we propose improved GPC mappings as well as new approaches for GPC cascading and binding for Xilinx FPGAs. With these improvements, we develop an integer linear programming (ILP) method for FPGA synthesis of GPC-based compressor trees that supports cascading and binding between GPCs. Experimental results show that the single-cycle compressor trees produced by the proposed ILP can reduce the average area by 42.40% compared with those generated by existing heuristic method, but are 13.16% slower; the pipelined compressor trees produced by the proposed ILP can reduce the average area by 33.43% at the cost of an average 14.35% decrease in maximum clock frequency compared with those obtained by existing heuristic method.

The three-dimensional (3D) carbon nanostructures/foams are commonly used as active materials for the high-performance flexible piezoresistive sensors due to their superior properties. However, the intrinsic brittleness and poor sensing properties of monolithic carbon material still limits its application. Rational design of the microstructure is an attractive approach to achieve piezoresistive material with superior mechanical and sensing properties, simultaneously. Herein, we introduce novel three-dimensional buckling carbon nanofibers (3D BCNFs) that feature a unique serpentine-buckling microstructure. The obtained 3D BCNFs exhibit superior mechanical properties, including super-elasticity (recovery speed up to 950 mm s−1), excellent flexibility (multiple folds), high compressibility (compressed by 90%), and high fatigue resistance (10,000 bending cycles). The pressure sensor fabricated by the 3D BCNFs shows a high sensitivity of 714.4 kPa−1, a fast response time of 23 ms, and a broad measuring range of 120 kPa. The pressure sensor is further applied to monitor the physiological signals of humans, and is capable of detecting the characteristic pulse waves from the radial artery, fingertip artery, and human-breath, respectively.

In this Letter, the authors propose a novel attention mechanism combined with a classical generative adversarial network (GAN) model to improve the visual quality of generated samples. This novel attention model is named regional attention GAN. The proposed mechanism can build dependencies between the high‐level representations extracted from attention regions of real images and corresponding feature maps of the generative network. By modelling these dependencies, the generative network can be facilitated to learn feature mapping and fit the distribution of real data. They conduct extensive experiments on widely used datasets CIFAR‐10, STL‐10, and CelebA. The quantitative and qualitative performance improvement over state‐of‐the‐art methods demonstrates the validity of the proposed attention mechanism in improving the quality of generated images.

This paper presents our first design of the on-chip EMI sensor array. The feedback signal from the sensor array helps to determine the location of the failed circuit on the chip of an integrated circuit (IC) when external electromagnetic interference (EMI) is applied to the IC. The array structure, the feedback signal and the corresponding circuit of the sensor cell is developed. The design is implemented with a FPGA. The functionality of the design is checked through measuring the generated feedback signal of the FPGA. The feedback signal suffers instability problems due to the on-chip process variation. A set of equations are developed to describe the performance limitation that current IC technology put on the sensor array. The trade-off between the timing and the spatial resolutions of the array is analyzed. The conclusion of the paper shows the necessary conditions to make the measurement method practical.

Oscillators are sensitive to radio frequency interference. Therefore they can be applied to measure radio frequency interference. Radio frequency interference sources cover wide frequency ranges. It is interesting to see the frequency dependence of the oscillator susceptibility to radio frequency interference. This paper uses the crystal oscillator as the device under test and performs a systematic simulation on susceptibility. The transistor-level simulation shows that the oscillator susceptibility varies strongly with the signal frequency. The behavior of the oscillator susceptibility could be characterized with critical features and explained with three response modes. Both the simulated behavior and the proposed response mechanism are confirmed with the measurement results. The conclusion of the paper helps to calibrate oscillator sensors. It is also an important guideline for building analysis model in the design phase of the sensor.

In this work, we report on the design of a wide-band digital lock-in amplifier (DLIA) of up to 65 MHz and its application for electrical impedance measurements in microfluidic devices. The DLIA is comprised of several dedicated technologies. First, it features a fully differential analog circuit, which includes a preamplifier with a low input noise of 4.4 nV/√Hz, a programmable-gain amplifier with a gain of 52 dB, and an anti-aliasing, fully differential low-pass filter with −76 dB stop-band attenuation. Second, the DLIA has an all-digital phase lock loop, which features a phase deviation of less than 0.02° throughout the frequency range. The phase lock loop utilizes an equally accurate period-frequency measurement, with a sub-ppm precision of frequency detection. Third, a modified clock link is implemented in the DLIA to improve the signal-to-noise ratio of the analog-to-digital converter affected by clock jitter of up to 20 dBc. A series of measurements were performed to characterize the DLIA, and the results showed an accurate performance. Additionally, impedance measurements of standard-size microparticles were performed by frequency sweep from 300 kHz to 30 MHz, using the DLIA in a microfluidic device. Different diameters of microparticle could be accurately distinguished according to the relative impedance at 2.5 MHz. The results confirm the promising applications of the DLIA in microfluidic electrical impedance measurements.

The localization of eye centers is a very useful cue for numerous applications like face recognition, facial expression recognition, and the early screening of neurological pathologies. Several methods relying on available light for accurate eye-center localization have been exploited. However, despite the considerable improvements that eye-center localization systems have undergone in recent years, only few of these developments deal with the challenges posed by the profile (non-frontal face). In this paper, we first use the explicit shape regression method to obtain the rough location of the eye centers. Because this method extracts global information from the human face, it is robust against any changes in the eye region. We exploit this robustness and utilize it as a constraint. To locate the eye centers accurately, we employ isophote curvature features, the accuracy of which has been demonstrated in a previous study. By applying these features, we obtain a series of eye-center locations which are candidates for the actual position of the eye-center. Among these locations, the estimated locations which minimize the reconstruction error between the two methods mentioned above are taken as the closest approximation for the eye centers locations. Therefore, we combine explicit shape regression and isophote curvature feature analysis to achieve robustness and accuracy, respectively. In practical experiments, we use BioID and FERET datasets to test our approach to obtaining an accurate eye-center location while retaining robustness against changes in scale and pose. In addition, we apply our method to non-frontal faces to test its robustness and accuracy, which are essential in gaze estimation but have seldom been mentioned in previous works. Through extensive experimentation, we show that the proposed method can achieve a significant improvement in accuracy and robustness over state-of-the-art techniques, with our method ranking second in terms of accuracy. According to our implementation on a PC with a Xeon 2.5Ghz CPU, the frame rate of the eye tracking process can achieve 38 Hz.