Search

Varying density of point clouds increases the difficulty of 3D detection. In this paper, we present a context-aware dynamic network (CADNet) to capture the variance of density by considering both point context and semantic context. Point-level contexts are generated from original point clouds to enlarge the effective receptive filed. They are extracted around the voxelized pillars based on our extended voxelization method and processed with the context encoder in parallel with the pillar features. With a large perception range, we are able to capture the variance of features for potential objects and generate attentive spatial guidance to help adjust the strengths for different regions. In the region proposal network, considering the limited representation ability of traditional convolution where same kernels are shared among different samples and positions, we propose a decomposable dynamic convolutional layer to adapt to the variance of input features by learning from the local semantic context. It adaptively generates the position-dependent coefficients for multiple fixed kernels and combines them to convolve with local features. Based on our dynamic convolution, we design a dual-path convolution block to further improve the representation ability. We conduct experiments on KITTI dataset and the proposed CADNet has achieved superior performance of 3D detection outperforming SECOND and PointPillars by a large margin at the speed of 30 FPS.

Due to their high photoelectric conversion efficiency (PCE) and low-cost fabrication process, n-type silicon (n-Si)/poly(3, 4-ethylenedioxythiophene):poly(styrenesulfonate) (PEDOT:PSS) hybrid heter...

Discovering diverse significant regions (e.g., beaks and wings for some bird species) and extracting discriminative features from them is vitally important in fine-grained image recognition. Currently, the attention-based approaches present promising performance, which generally extract the fine-grained features by cropping or sampling significant parts. However, the cropping methods usually suffer from a fixed number of parts and difficulty to highlight irregular regions, and existing sampling methods may produce extremely distorted images. To effectively capture the fine-grained features, we propose an end-to-end separated smooth sampling network (SSSNet) in this paper. Specifically, we propose a separated smooth sampling module to highlight diverse significant regions of an image. Different from previous methods, we adopt smooth sampling on two separated coordinates to process images, which can effectively highlight discriminative contents and meanwhile avoid extreme distortion. We further propose an iterative masking method to embed into SSSNet, which can produce multiple attention maps without overlap to represent different significant regions. We conduct extensive experiments on CUB-200–2011, Stanford-Cars, and FGVC-Aircraft datasets. The results show the effectiveness of separated smooth sampling, and our SSSNet achieves better performance against previous state-of-the-art approaches under the same settings.

Carrier-selective contacts, especially in electron-selective contact (ESC), between the metal electrodes and the crystalline silicon (c-Si) substrates play a pivotal section in promoting the power ...

Temporal action detection is a challenging task due to vagueness of action boundaries. To tackle this issue, we propose an end-to-end progressive boundary refinement network (PBRNet) in this paper. PBRNet belongs to the family of one-stage detectors and is equipped with three cascaded detection modules for localizing action boundary more and more precisely. Specifically, PBRNet mainly consists of coarse pyramidal detection, refined pyramidal detection, and fine-grained detection. The first two modules build two feature pyramids to perform the anchor-based detection, and the third one explores the frame-level features to refine the boundaries of each action instance. In the fined-grained detection module, three frame-level classification branches are proposed to augment the frame-level features and update the confidence scores of action instances. Evidently, PBRNet integrates the anchor-based and frame-level methods. We experimentally evaluate the proposed PBRNet and comprehensively investigate the effect of the main components. The results show PBRNet achieves the state-of-the-art detection performances on two popular benchmarks: THUMOS'14 and ActivityNet, and meanwhile possesses a high inference speed.

Angiotensin-converting enzyme 2 (ACE2), a transmembrane protein, is the main entry point for certain coronaviruses including the new coronavirus SARS-CoV-2 to enter cells. Synthesizing the PET imaging probe Al18F-DX600-BCH which is high-affinity ACE2 is aim to detect the expression of ACE2 in body and monitor the therapeutic effect. The Al18F-DX600-BCH was obtained manually with a 20.4% ± 5.2% radiochemical yield without attenuation correction and an over 99% purified radiochemical purity, being stable in vitro within 4 hours and cleared rapidly in blood (the half-lives of the distribution phase and clearance phase were 2.12 min and 25.31 min, respectively). Results of both biodistribution and PET imaging showed that Al18F-DX600-BCH was highly accumulated in the kidney (SUVkidney/normal > 50), and specific uptake in testis (SUVtestis/normal > 10) was observed in rat images. The kidney (++), gastrointestinal (++) and bronchial (+++) cells were evidenced of ACE2 positive by IHC staining of rats. A total of 10 volunteers were enrolled and received PET/CT 1 hour and 2 hours after injection or dynamic PET/CT during 0-330 seconds (NCT04542863), from which strong radioactivity accumulation was mostly observed in the genitourinary system (SUVrenal cortex = 32.00, SUVtestis = 4.56), and moderate accumulation in conjunctiva and nasal mucosa for several cases. This work firstly reported the probe Al18F-DX600-BCH targeting ACE2, conducting preliminary preclinical experiments and a total of 10 clinical transformations, which demonstrated the potential and possibility of non-invasive mapping of ACE2. Trial registration: ClinicalTrials.gov NCT04542863. Registered 9 September 2020.

Electron-transport layer (ETL) that promotes charge carrier separation and electron extraction is a key component of perovskite solar cells (PSCs). Here, we report a simple approach for improving c...

Irrespective of the success on reduction of contact resistivity, lack of chemical passivation of evaporated metal oxides heavily hinders their applications as passivating contacts, such contacts can be an alternative route for high efficiency and cost effective silicon solar cells. Here, we demonstrate that electron beam evaporated magnesium oxide (MgOx) thin film can work as a promising electron-selective passivating contact for n-Si solar cells after a post-annealing treatment and an alumina-initiated atomic hydrogenation. 10 nm MgOx on n-Si provided a surface recombination velocity down to 14.9 cm/s while 1 nm MgOx showed a low contact resistivity of 14 mΩ cm2. Comprehensive characterizations revealed the formation of Si–O–Mg bonds and the activation of atomic hydrogens were the main reasons for such high-level passivation. A PERC-like dopant-free rear contact was formed by using the 1 nm-MgOx as electron-collector and the 10 nm-MgOx as passivating layer, the resultant solar cells achieved 27% increment in efficiency and 51 mV increase in open-circuit voltage in comparison with reference devices. The ways of improving passivation quality of MgOx and novel design of contact structure open up the possibility of using evaporation-processed metal oxides as effective and low-cost carrier-selective passivating contacts for n-Si photovoltaic devices.

Feedback is a fundamental mechanism existing in the human visual system, but has not been explored deeply in designing computer vision algorithms. In this paper, we claim that feedback plays a critical role in understanding convolutional neural networks (CNNs), e.g., how a neuron in CNNs describes an object's pattern, and how a collection of neurons form comprehensive perception to an object. To model the feedback in CNNs, we propose a novel model named Feedback CNN and develop two new processing algorithms, i.e., neural pathway pruning and pattern recovering. We mathematically prove that the proposed method can reach local optimum. Note that Feedback CNN belongs to weakly supervised methods and can be trained only using category-level labels. But it possesses a powerful capability to accurately localize and segment category-specific objects. We conduct extensive visualization analysis, and the results reveal the close relationship between neurons and object parts in Feedback CNN. Finally, we evaluate the proposed Feedback CNN over the tasks of weakly supervised object localization and segmentation, and the experimental results on ImageNet and Pascal VOC show that our method remarkably outperforms the state-of-the-art ones.

At present, it is a significant challenge to design electrode materials with both desirable electrochemical performance and high mass loading for supercapacitors by using a simple, efficient and safe method. With the introduction of Ni2+ ions, the hollow Ni-decorated Co9S8 nanospheres meeting the above requirements have been successfully synthesized by one-step electrodeposition method under a two-electrode system. As electrodes for supercapacitors, such Ni-decorated Co9S8 electrodes deliver an ultrahigh areal and volumetric specific capacitance of 5.64 F cm−2 and 171.85 F cm−3 at a current density of 1 mA cm−2, superior rate capability (88.9% retention at a current density of 20 mA cm−2) and remarkable cycling stability with a capacitance retention of 89% after 8500 cycles. Also, the as-fabricated flexible quasi-solid-state asymmetric supercapacitors based on Ni-decorated Co9S8 nanospheres and active carbon electrodes present high energy density. The practicability and operability of the device were successfully demonstrated by lighting up the LEDs with five different colors.

Black silicon (b-Si) with nanotextures is a promising light-trapping scheme for potentially achieving high conversion efficiency at reduced manufacturing cost in crystalline-silicon solar cells. However, the inherently high aspect-ratio and tiny feature size of the nanostructures are subject to severe surface (large surface areas) and Auger recombination (worse doping profile). These will abate the cost values of b-Si since one has to adopt a comprise strategy of applying shallow nanotextures with antireflection and passivation layers. Here, we show that silicon microwire solar cells featuring well-defined radial junctions can extensively suppress both surface and Auger recombination by providing excellent all-around electrical field. The radially doped silicon micropillar devices even show an internal quantum efficiency as good as that of planar substrate and their measured minority carrier lifetimes become nearly independent of total surface area. A great reduction in short-circuit current density loss was further identified as the junction abruptly changed from a fully diffused to a core-shell configuration, manifesting the powerful effectiveness of radial p-n+ junction on the suppression of Auger recombination. Furthermore, silicon microwire solar cell with a radial junction demonstrates 37% increase in efficiency compared with the reference cell, suggesting a feasible strategy towards high-efficiency solar devices.

Recent years have witnessed great progress of object detection. However, due to the domain shift problem, applying the knowledge of an object detector learned from one specific domain to another one often suffers severe performance degradation. Most existing methods adopt feature alignment either on the backbone network or instance classifier to increase the transferability of object detector. Differently, we propose to perform feature alignment in the RPN stage such that the foreground and background RPN proposals in target domain can be effectively distinguished. Specifically, we first construct one set of learnable RPN prototpyes, and then enforce the RPN features to align with the prototypes for both source and target domains. It essentially cooperates the learning of RPN prototypes and features to align the source and target RPN features. Particularly, we propose a simple yet effective method suitable for RPN feature alignment to generate high-quality pseudo label of proposals in target domain, i.e., using the filtered detection results with IoU. Furthermore, we adopt Grad CAM to find the discriminative region within a foreground proposal and use it to increase the discriminability of RPN features for alignment. We conduct extensive experiments on multiple cross-domain detection scenarios, and the results show the effectiveness of our proposed method against previous state-of-the-art methods.

Few-shot learning is a challenging task since only few instances are given for recognizing an unseen class. One way to alleviate this problem is to acquire a strong inductive bias via meta-learning on similar tasks. In this paper, we show that such inductive bias can be learned from a flat collection of unlabeled images, and instantiated as transferable representations among seen and unseen classes. Specifically, we propose a novel part-based self-supervised representation learning scheme to learn transferable representations by maximizing the similarity of an image to its discriminative part. To mitigate the overfitting in few-shot classification caused by data scarcity, we further propose a part augmentation strategy by retrieving extra images from a base dataset. We conduct systematic studies on miniImageNet and tieredImageNet benchmarks. Remarkably, our method yields impressive results, outperforming the previous best unsupervised methods by 7.74% and 9.24% under 5-way 1-shot and 5-way 5-shot settings, which are comparable with state-of-the-art supervised methods., Accepted by IJCAI 2021

Hepatocellular carcinoma (HCC) is characterized by a high rate of incidence and recurrence, and resistance to chemotherapy may aggravate the poor prognosis of HCC patients. Sorafenib resistance is a conundrum to the treatment of advanced/recurrent HCC. Therefore, studies on the molecular pathogenesis of HCC and the resistance to sorafenib are of great interest. Here, we report that GINS1 was highly expressed in HCC tumors, associated with tumor grades, and predicted poor patient survival using Gene Expression Omnibus (GEO) databases exploration. Cell cycle, cell proliferation assay and in vivo xenograft mouse model indicated that knocking down GINS1 induced in G1/S phase cell cycle arrest and decreased tumor cells proliferation in vitro and in vivo. Spheroid formation assay results showed that GINS1 promoted the stem cell activity of HCC tumor cells. Furthermore, GEO database (GSE17112) analysis showed that HRAS oncogenic gene set was enriched in GINS1 high-expressed cancer cells, and quantitative real-time PCR, and Western blot results proved that GINS1 enhanced HCC progression through regulating HRAS signaling pathway. Moreover, knocking down endogenous GINS1 with shGINS1 increased the sensitivity of HCC cells to sorafenib, and restoring HRAS or stem associated pathway partly recovered the sorafenib resistance. Overall, the collective findings highlight GINS1 functions in hepatocarcinogenesis and sorafenib resistance, and indicate its potential use of GINS1 in drug-resistant HCC.

Purpose To develop generative adversarial networks (GANs) that synthesize realistic anterior segment optical coherence tomography (AS-OCT) images and evaluate deep learning (DL) models that are trained on real and synthetic datasets for detecting angle closure. Methods The GAN architecture was adopted and trained on the dataset with AS-OCT images collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, synthesizing open- and closed-angle AS-OCT images. A visual Turing test with two glaucoma specialists was performed to assess the image quality of real and synthetic images. DL models, trained on either real or synthetic datasets, were developed. Using the clinicians' grading of the AS-OCT images as the reference standard, we compared the diagnostic performance of open-angle vs. closed-angle detection of DL models and the AS-OCT parameter, defined as a trabecular-iris space area 750 µm anterior to the scleral spur (TISA750), in a small independent validation dataset. Results The GAN training included 28,643 AS-OCT anterior chamber angle (ACA) images. The real and synthetic datasets for DL model training have an equal distribution of open- and closed-angle images (all with 10,000 images each). The independent validation dataset included 238 open-angle and 243 closed-angle AS-OCT ACA images. The image quality of real versus synthetic AS-OCT images was similar, as assessed by the two glaucoma specialists, except for the scleral spur visibility. For the independent validation dataset, both DL models achieved higher areas under the curve compared with TISA750. Two DL models had areas under the curve of 0.97 (95% confidence interval, 0.96-0.99) and 0.94 (95% confidence interval, 0.92-0.96). Conclusions The GAN synthetic AS-OCT images appeared to be of good quality, according to the glaucoma specialists. The DL models, trained on all-synthetic AS-OCT images, can achieve high diagnostic performance. Translational relevance The GANs can generate realistic AS-OCT images, which can also be used to train DL models.

Image inpainting faces the challenging issue of the requirements on structure reasonableness and texture coherence. In this paper, we propose a two-stage inpainting framework to address this issue. The basic idea is to address the two requirements in two separate stages. Completed segmentation of the corrupted image is firstly predicted through segmentation reconstruction network, while fine-grained image details are restored in the second stage through an image generator. The two stages are connected in series as the image details are generated under the guidance of completed segmentation map that predicted in the first stage. Specifically, in the second stage, we propose a novel graph-based relation network to model the relationship existed in corrupted image. In relation network, both intra-relationship for pixels in the same semantic region and inter-relationship between different semantic parts are considered, improving the consistency and compatibility of image textures. Besides, contrastive loss is designed to facilitate the relation network training. Such a framework not only simplifies the inpainting problem directly, but also exploits the relationship in corrupted image explicitly. Extensive experiments on various public datasets quantitatively and qualitatively demonstrate the superiority of our approach compared with the state-of-the-art.

Recent image inpainting works have shown promising results thanks to great advances of generative adversarial networks (GANs). However, these methods would still generate distorted structures or blurry textures for the situation of large missing area, which is mainly due to the inherent difficulty to train GANs. In this paper, we propose a novel multi-level discriminator (MLD) and wavelet loss (WT) to improve the learning of image inpainting generators. Our method does not change the structure of generator and only works in the training phase, which thus can be easily embedded into sophisticated inpainting networks and would not increase the inference time. Specifically, MLD divides the mask into multiple subregions and then imposes an independent discriminator to each subregion. It essentially increases the distribution overlap between the real images and generated images. Consequently, MLD improves the optimization of GANs by providing more effective gradients to generators. In addition, WT builds a reconstruction loss in the frequency domain, which can facilitate the training of image inpainting networks as a regularization term. Consequently, WT can enforce the generated contents to be more consistent and sharper than the traditional pixel-wise reconstruction loss. We integrate WLD and WT into off-the-shelf image inpainting networks, and conduct extensive experiments on CelebA-HQ, Paris StreetView, and Places2. The results well demonstrate the effectiveness of the proposed method, which achieves state-of-the-art performance and generates higher-quality images than the baselines.

The requirement to recognize the iris image of low-quality is rapidly increasing with the practical application of iris recognition, especially the urgent need for high-throughput or applications in covert situations. The appr-circle fitting can not meet the needs due to the high time cost and non-accurate boundary estimation during the normalization process. Furthermore, the appr-circular hypothesis of iris and pupil is not entirely established due to the squint and occlusion in non-cooperative environments. To mitigate this problem, a multi-mask normalization without appr-circular parameter estimation is proposed to make full use of the segmented masks, which provide robust pixel-level iris boundaries. It bridges the segmentation and feature extraction to recognize the low-quality iris, which is thrown directly by the traditional methods. Thus, the complex samples with no appr-circular iris or massive occlusions can be recognized correctly. The extensive experiments are conducted on the representative and challenging databases to verify the generalization and the accuracy of the proposed iris normalization method. Besides, the throughput rate is significantly improved.

Person re-identification aims at identifying a certain pedestrian across non-overlapping multi-camera networks in different time and places. Existing person re-identification approaches mainly focus on matching pedestrians on images; however, little attention has been paid to re-identify pedestrians in videos. Compared to images, video clips contain motion patterns of pedestrians, which is crucial to person re-identification. Moreover, consecutive video frames present pedestrian appearance with different body poses and from different viewpoints, providing valuable information toward addressing the challenge of pose variation, occlusion, and viewpoint change, and so on. In this article, we propose a Dense 3D-Convolutional Network (D3DNet) to jointly learn spatio-temporal and appearance representation for person re-identification in videos. The D3DNet consists of multiple three-dimensional (3D) dense blocks and transition layers. The 3D dense blocks enlarge the receptive fields of visual neurons in both spatial and temporal dimensions, leading to discriminative appearance representation as well as short-term and long-term motion patterns of pedestrians without the requirement of an additional motion estimation module. Moreover, we formulate a loss function consisting of an identification loss and a center loss to minimize intra-class variance and maximize inter-class variance simultaneously, toward addressing the challenge of large intra-class variance and small inter-class variance. Extensive experiments on two real-world video datasets of person identification, i.e., MARS and iLIDS-VID, have shown the effectiveness of the proposed approach.

For crystalline-silicon (c-Si) solar cells, the interdigitated back contact (IBC) structure has been long known as an efficient way to approach the theoretical limit of efficiency. However, the complexity of fabricating this kind of devices as well as the high dependence on expensive vacuum systems pose concerns about their commercial potential. Here, we demonstrate a novel c-Si IBC solar cell featuring dopant-free heterocontacts for both polarities, i.e. a solution-proceeded PEDOT:PSS film as hole-transporting layer (HTL) and an evaporated magnesium-oxide film as electron-transporting layer (ETL). Our innovatively buried ETL method provides substantial simplification on the architecture and fabrication of the IBC cells and makes it possible to adapt solution-proceeded HTLs while keeping good passivation in gap regions. The IBC solar cell shows an efficiency of 16.3%, with a promising short-circuit current density (Jsc) up to 38.4 mA/cm2. A thorough simulation concerning the influence of pitch size, surface recombination rate (at ETL and gap regions) was conducted, revealing a readily achievable Jsc of 41 mA/ cm 2 and a PCE beyond 22%. Our findings demonstrated a feasibility of using solution method to fabricate high efficiency dopant-free IBC solar cells.

This paper addresses the issue on how to more effectively coordinate the depth with RGB aiming at boosting the performance of RGB-D object detection. Particularly, we investigate two primary ideas under the CNN model: property derivation and property fusion. Firstly, we propose that the depth can be utilized not only as a type of extra information besides RGB but also to derive more visual properties for comprehensively describing the objects of interest. Then a two-stage learning framework consisting of property derivation and fusion is constructed. Here the properties can be derived either from the provided color/depth or their pairs (e.g. the geometry contour). Secondly, we explore the fusion methods of different properties in feature learning, which is boiled down to, under the CNN model, from which layer the properties should be fused together. The analysis shows that different semantic properties should be learned separately and combined before passing into the final classifier. Actually, such a detection way is in accordance with the mechanism of the primary visual cortex (V1) in brain. We experimentally evaluate the proposed method on the challenging datasets NYUD2 and SUN RGB-D, and both achieve remarkable performances that outperform the baselines.

Nanostructured silicon (Si) can provide improved light trapping capacity in Si/organic hybrid solar cells (HSCs) due to its low light reflectance compared with planar Si. However, the poor contact of nanostructured Si/organic interface and serious recombination on the uncovered Si surface can result in an inferior open-circuit voltage ( V OC ) and fill factor ( FF ) for HSCs. Here, we have designed and fabricated a novel cone-shaped Si nanoholes (SiNHs) structures to form a superior interface contact between SiNHs and organic layer by using an advanced metal assisted chemical etching method. In addition, a Cs 2 CO 3 layer was also introduced on the Si back surface to reduce the contact resistance as well as to suppress the surface recombination for the HSCs. With such improvements, a power conversion efficiency (PCE) up to 13.5% was achieved for Si/organic hybrid solar cell with the back passivation layer. This work provided a new strategy to improve the junction quality and performance of nanostructured Si/PEDOT:PSS HSCs.

The optical and electrical properties of PEDOT:PSS organic layer play a very important role in determining the power conversion efficiency (PCE) of Si–organic hybrid solar cells (HSCs). In the present study, properties of PEDOT:PSS thin films with reduced graphene oxide (rGO) and their impacts on the performances of the resultant Si–organic HSCs have been systematically investigated. The electrical conductivity of PEDOT:PSS improved 35% when rGO was added to PEDOT:PSS, and the fabricated HSCs with 2 mg/ml rGO addition yielded an PCE of 11.95% with a J sc of 31.94 mA cm −2 , a V oc of 579 mV and a FF of 0.648 . However, excess rGO would deteriorate the solar cells performances and it might create additional defects and prevent carriers being collected. The Raman spectroscopy, sheet resistance and EQE analyses with rGO suggested that the interaction between the conductive rGO flakes and the aromatic PEDOT most probably not only provide additional charge transport pathways in hole transport layer to improve carrier mobility leading to a higher carrier collection efficiency, but also suppress the electron recombination at the junction interface. In addition, the rGO serve as an antireflection coating to reduce the reflectance of PEDOT:PSS thin film leading to further enhanced performances of solar cells.

The background information is a significant prior for salient object detection, especially when images contain cluttered background and diverse object parts. In this paper, we propose a background-driven salient object detection (BD-SOD) method to more comprehensively exploit the background prior, aiming at generating more accurate and robust salient maps. To be specific, we first exploit the background prior to conduct the saliency estimation, i.e., computing the regional saliency values. In this stage, the background prior is utilized in threefold: restricting the reference regions to only the background regions, weighting the contribution of reference regions, and leveraging the importance of different features. Benefiting from such an explicit utilization, the proposed model can greatly mitigate the negative interference of the cluttered background and diverse object parts. We then embed the background prior into the optimization graph for saliency refinement. Specifically, two virtual supernodes (representing the background and foreground, respectively) are introduced with extra connections, and the nonlocal feature connections between similar regions are also set up. These connections enhance the power of optimization graph to alleviate the perturbations from diverse parts, and thus help to achieve the uniformity of saliency values. Finally, we provide systematical studies to investigate the effectiveness of the proposed BD-SOD in exploiting the valuable background prior. Experimental results on multiple public benchmark datasets, including MSRA-1000, THUS-10000, PASCAL-S, and ECSSD, clearly show that BD-SOD consistently outperforms the well-established baselines and achieves state-of-the-art performance.

The problem of adaptive multi-user scalable layered video transmission is considered in energy harvesting (EH) aided wireless communication systems. With the goal of improving the quality of video services while providing fairness amongst the users despite the random nature of both energy harvesting and the channel quality, we formulate our Scalable Video Coding (SVC) design as a Constrained Utility Function Maximization (CUFM) problem. The proportional fairness and playback smoothness of our design is guaranteed by maximizing the log-sum of the users’ video qualities, while satisfying the battery fullness constraint and video layer (quality) fluctuation constraint. By invoking the classical Lyapunov drift based optimization technique, we further decompose the CUFM problem into two parallel subproblems, i.e., a dynamic transmission power allocation problem and a dynamic layer selection problem. By solving these two subproblems, we derive a joint power allocation and video layer selection strategy for multi-user SVC video transmission. The theoretical performance bound of the proposed solution is also presented. Numerical simulations are conducted with real H.264 SVC video traces and the experimental results demonstrate the reduced playback interruption rate and layer switching rate compared to a heuristic algorithm ProNTO. The results also illustrate a tradeoff between the system’s utility function and the playback smoothness experienced by the users.

Video semantic segmentation is active in recent years benefited from the great progress of image semantic segmentation. For such a task, the per-frame image segmentation is generally unacceptable in practice due to high computation cost. To tackle this issue, many works use the flow-based feature propagation to reuse the features of previous frames. However, the optical flow estimation inevitably suffers inaccuracy and then causes the propagated features distorted. In this paper, we propose distortion-aware feature correction to alleviate the issue, which improves video segmentation performance by correcting distorted propagated features. To be specific, we firstly propose to transfer distortion patterns from feature into image space and conduct effective distortion map prediction. Benefited from the guidance of distortion maps, we proposed Feature Correction Module (FCM) to rectify propagated features in the distorted areas. Our proposed method can significantly boost the accuracy of video semantic segmentation at a low price. The extensive experimental results on Cityscapes and CamVid show that our method outperforms the recent state-of-the-art methods., Comment: 12 pages, 16 figures. The paper has been accepted by IEEE Transactions on Circuits and Systems for Video Technology(TCSVT) 2020

Conventionally, deep neural networks are trained offline, relying on a large dataset prepared in advance. This paradigm is often challenged in real-world applications, e.g. online services that involve continuous streams of incoming data. Recently, incremental learning receives increasing attention, and is considered as a promising solution to the practical challenges mentioned above. However, it has been observed that incremental learning is subject to a fundamental difficulty -- catastrophic forgetting, namely adapting a model to new data often results in severe performance degradation on previous tasks or classes. Our study reveals that the imbalance between previous and new data is a crucial cause to this problem. In this work, we develop a new framework for incrementally learning a unified classifier, e.g. a classifier that treats both old and new classes uniformly. Specifically, we incorporate three components, cosine normalization, less-forget constraint, and inter-class separation, to mitigate the adverse effects of the imbalance. Experiments show that the proposed method can effectively rebalance the training process, thus obtaining superior performance compared to the existing methods. On CIFAR-100 and ImageNet, our method can reduce the classification errors by more than 6% and 13% respectively, under the incremental setting of 10 phases.

Recent research on super-resolution has achieved great success due to the development of deep convolutional neural networks (DCNNs). However, super-resolution of arbitrary scale factor has been ignored for a long time. Most previous researchers regard super-resolution of different scale factors as independent tasks. They train a specific model for each scale factor which is inefficient in computing, and prior work only take the super-resolution of several integer scale factors into consideration. In this work, we propose a novel method called Meta-SR to firstly solve super-resolution of arbitrary scale factor (including non-integer scale factors) with a single model. In our Meta-SR, the Meta-Upscale Module is proposed to replace the traditional upscale module. For arbitrary scale factor, the Meta-Upscale Module dynamically predicts the weights of the upscale filters by taking the scale factor as input and use these weights to generate the HR image of arbitrary size. For any low-resolution image, our Meta-SR can continuously zoom in it with arbitrary scale factor by only using a single model. We evaluated the proposed method through extensive experiments on widely used benchmark datasets on single image super-resolution. The experimental results show the superiority of our Meta-Upscale., Comment: 10 pages, 4 figures

Background Semi-supervised learning algorithms can leverage an unlabeled dataset when labeling is limited or expensive to obtain. In the current study, we developed and evaluated a semi-supervised generative adversarial networks (GANs) model that detects closed-angle on anterior segment optical coherence tomography (AS-OCT) images using a small labeled dataset. Methods In this cross-sectional study, a semi-supervised GANs model was developed for automatic closed-angle detection training on a small labeled and large unsupervised training dataset collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong (JSIEC). The closed-angle was defined as iris-trabecular contact beyond the scleral spur in AS-OCT images. We further developed two supervised deep learning (DL) models training on the same supervised dataset and the whole dataset separately. The semi-supervised GANs model and supervised DL models' performance were compared on two independent testing datasets from JSIEC (515 images) and the Department of Ophthalmology (84 images), National University Health System, respectively. The diagnostic performance was assessed by evaluation matrices, including the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Results For closed-angle detection using clinician grading of AS-OCT imaging as the reference standard, the semi-supervised GANs model showed comparable performance, with AUCs of 0.97 (95% CI, 0.96-0.99) and 0.98 (95% CI, 0.94-1.00), compared with the supervised DL model (using the whole dataset) [AUCs of 0.97 (95% CI, 0.96-0.99), and 0.97 (95% CI, 0.94-1.00)]. When training on the same small supervised dataset, the semi-supervised GANs achieved performance at least as well as, if not better than, the supervised DL model [AUCs of 0.90 (95% CI: 0.84-0.96), and 0.92 (95% CI, 0.86-0.97)]. Conclusions The semi-supervised GANs method achieves diagnostic performance at least as good as a supervised DL model when trained on small labeled datasets. Further development of semi-supervised learning methods could be useful within clinical and research settings. Trial registration number ChiCTR2000037892.

3D petal-like NiCo2S4 nanostructures have been fabricated via a simple, mild and efficient hydrothermal strategy and the growth mechanism of NiCo2S4 nano-petals has been investigated. Such NiCo2S4 nano-petal electrodes can deliver an ultrahigh specific capacitance of 2036.5 F g−1 at a current density of 1 A g−1, superior rate capability and remarkable cycle stability (94.3% of capacitance retention after 5000 cycles). The as-fabricated asymmetric supercapacitors based on NiCo2S4 nano-petals//active carbon electrodes demonstrate a high energy density of 35.6 W h kg−1 at a power density of 819.5 W kg−1, with both long-term cycling and high rate stabilities. Such supercapacitors have been tested to power ten LEDs (2.03 V, 20 mA) in series for around 60 minutes, indicating their great potential for practical application.

In this paper, we address the issue of recognizing human actions with low observational latency, which is vital for many applications such as virtual reality and interactive entertainment. Then our purpose is to achieve competitive action recognition performance from very short video clips. Such a task essentially is challenging because only very limited information is provided. To this end, we first develop a feature extraction method to exploit both motion (local flow) and appearance (local shape) features such that the information insufficiency can be effectively mitigated. Then we propose an action representation method named Part Movement Model (PMM), which explicitly captures the spatial-temporal structure of human actions and divides the actions into discriminative part movements. Consequently, the actions can be better represented and the competitive performance can be achieved although only short clips are used. Finally, we experimentally verify the effectiveness of the proposed method on three benchmark datasets. The results show that short clips of 6−7 frames (0.2−0.3 second video) are enough to achieve the recognition performance comparable to the baselines with high latency.

Salient object detection aims to identify both spatial locations and scales of the salient object in an image. However, previous saliency detection methods generally fail in detecting the whole objects, especially when the salient objects are actually composed of heterogeneous parts. In this work, we propose a saliency bias and diffusion method to effectively detect the complete spatial support of salient objects. We first introduce a novel saliency-aware feature to bias the objectness detection for saliency detection on a given image and incorporate the saliency clues explicitly in refining the saliency map. Then, we propose a saliency diffusion method to fuse the saliency confidences of different parts from the same object for discovering the whole salient object, which uses the learned visual similarities among object regions to propagate the saliency values across them. Benefiting from such bias and diffusion strategy, the performance of salient object detection is significantly improved, as shown in the comprehensive experimental evaluations on four benchmark data sets, including MSRA-1000, SOD, SED, and THUS-10000.

Purpose This study implements and demonstrates a deep learning (DL) approach for screening referable horizontal strabismus based on primary gaze photographs using clinical assessments as a reference. The purpose of this study was to develop and evaluate deep learning algorithms that screen referable horizontal strabismus in children's primary gaze photographs. Methods DL algorithms were developed and trained using primary gaze photographs from two tertiary hospitals of children with primary horizontal strabismus who underwent surgery as well as orthotropic children who underwent routine refractive tests. A total of 7026 images (3829 non-strabismus from 3021 orthoptics [healthy] subjects and 3197 strabismus images from 2772 subjects) were used to develop the DL algorithms. The DL model was evaluated by 5-fold cross-validation and tested on an independent validation data set of 277 images. The diagnostic performance of the DL algorithm was assessed by calculating the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Results Using 5-fold cross-validation during training, the average AUCs of the DL models were approximately 0.99. In the external validation data set, the DL algorithm achieved an AUC of 0.99 with a sensitivity of 94.0% and a specificity of 99.3%. The DL algorithm's performance (with an accuracy of 0.95) in diagnosing referable horizontal strabismus was better than that of the resident ophthalmologists (with accuracy ranging from 0.81 to 0.85). Conclusions We developed and evaluated a DL model to automatically identify referable horizontal strabismus using primary gaze photographs. The diagnostic performance of the DL model is comparable to or better than that of ophthalmologists. Translational relevance DL methods that automate the detection of referable horizontal strabismus can facilitate clinical assessment and screening for children at risk of strabismus.

Dopant-free heterojunction opens new doors to highly efficient silicon solar cells with interdigitated back-contacts (IBC) via an easy hard-mask processing. However, the existence of inevitable overlap between the hole- and electron-transport layers may cause edge leakage and recombination, which will deteriorate the power conversion efficiency. Here we unambiguously determined the edge recombination and recombination losses quantitatively, in combination with detailed comparisons in photovoltaic parameters, dark and light current-voltage (I–V) curves, partially illuminated I–V curves, of the hard-mask processed and the lithography processed IBC devices. Without the interfacial passivation layer, the solar cells fabricated by the hard-mask method suffer severe edge recombination with loss of 3 × 10−4 A and a quite poor fill factor (FF) of ~66%, suggesting that the edge recombination could be another important issue affecting the FF besides the series resistance. With the clear understanding of the edge effect, we finely control the edge overlap, and finally obtained silicon dopant-free solar cells (with of intrinsic amorphous silicon as passivation layer) with over 20% efficiency and 73% FF either by lithography or by hard-mask methods.

Temporal action detection in untrimmed videos is an important and challenging visual task. State-of-the-art works always adopt a multi-stage pipeline, i.e., a class-agnostic segment proposal followed by a multi-label action classification. This pipeline is computationally slow and hard to optimize as each stage need be trained separately. Moreover, a desirable method should go beyond segment-level localization and make dense predictions with precise boundaries. We introduce a novel detection model in this paper, Single-stage Multi-location Convolutional Network (SMC), which completely eliminates the proposal generation and spatio-temporal feature resampling, and predicts frame-level action locations with class probabilities in a unified end-to-end network. Specifically, we associate a set of multi-scale default locations with each feature map cell in multiple layers, then predict the location offsets to the default locations, as well as action categories. SMC in practice is faster than the existing methods (753 FPS on a Titan X Maxwell GPU) and achieves state-of-the-art performance on THUMOS’14 and MEXaction2.

License plate recognition (LPR) is a fundamental component of various intelligent transport systems, which is always expected to be accurate and efficient enough. In this paper, we propose a novel LPR framework consisting of semantic segmentation and character counting, towards achieving human-level performance. Benefiting from innovative structure, our method can recognize a whole license plate once rather than conducting character detection or sliding window followed by per-character recognition. Moreover, our method can achieve higher recognition accuracy due to more effectively exploiting global information and avoiding sensitive character detection, and is time-saving due to eliminating one-by-one character recognition. Finally, we experimentally verify the effectiveness of the proposed method on two public datasets (AOLP and Media Lab) and our License Plate Dataset. The results demonstrate our method significantly outperforms the previous state-of-the-art methods, and achieves the accuracies of more than 99% for almost all settings.

In this work we propose a novel framework named Dual-Net aiming at learning more accurate representation for image recognition. Here two parallel neural networks are coordinated to learn complementary features and thus a wider network is constructed. Specifically, we logically divide an end-to-end deep convolutional neural network into two functional parts, i.e., feature extractor and image classifier. The extractors of two subnetworks are placed side by side, which exactly form the feature extractor of DualNet. Then the two-stream features are aggregated to the final classifier for overall classification, while two auxiliary classifiers are appended behind the feature extractor of each subnetwork to make the separately learned features discriminative alone. The complementary constraint is imposed by weighting the three classifiers, which is indeed the key of DualNet. The corresponding training strategy is also proposed, consisting of iterative training and joint finetuning, to make the two subnetworks cooperate well with each other. Finally, DualNet based on the well-known CaffeNet, VGGNet, NIN and ResNet are thoroughly investigated and experimentally evaluated on multiple datasets including CIFAR-100, Stanford Dogs and UEC FOOD-100. The results demonstrate that DualNet can really help learn more accurate image representation, and thus result in higher accuracy for recognition. In particular, the performance on CIFAR-100 is state-of-the-art compared to the recent works.

In this paper, we propose a novel domain-specific dataset named VegFru for fine-grained visual categorization (FGVC). While the existing datasets for FGVC are mainly focused on animal breeds or man-made objects with limited labelled data, VegFru is a larger dataset consisting of vegetables and fruits which are closely associated with the daily life of everyone. Aiming at domestic cooking and food management, VegFru categorizes vegetables and fruits according to their eating characteristics, and each image contains at least one edible part of vegetables or fruits with the same cooking usage. Particularly, all the images are labelled hierarchically. The current version covers vegetables and fruits of 25 upper-level categories and 292 subordinate classes. And it contains more than 160,000 images in total and at least 200 images for each subordinate class. Accompanying the dataset, we also propose an effective framework called HybridNet to exploit the label hierarchy for FGVC. Specifically, multiple granularity features are first extracted by dealing with the hierarchical labels separately. And then they are fused through explicit operation, e.g., Compact Bilinear Pooling, to form a unified representation for the ultimate recognition. The experimental results on the novel VegFru, the public FGVC-Aircraft and CUB-200-2011 indicate that HybridNet achieves one of the top performance on these datasets. The dataset and code are available at https://github.com/ustc-vim/vegfru.

Recently, deep learning methods have been extensively applied for action recognition in videos. Most existing deep networks equally treat every video frame and directly assign a video label to all the frames sampled from it. However, discriminative action may occurs sparsely in a few key frames in a video, and other frames are less relevant or even irrelevant to the action class. Equally treating all the frames will hurt performance. To address this issue, we propose a temporal attention model which learns to recognize human actions in videos while focusing selectively on the informative frames. Our model does not need explicit annotations regarding such informative frames during training and testing. Specifically, we adopt Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) unit and attaches higher importance to the frames which are discriminative for the task at hand. Our method consistently improves on no-attention methods, with both RGB and optical flow based deep ConvNets. We achieve state-of-the-art performance on two challenging datasets of UCF101 and HMDB51.

For streaming media systems based on cloud service, an efficient and adaptive resource scheduling is strongly required. In this paper, we aim to make the small-scale streaming edge cloud admit more requests such that lower perstream operation cost is achieved. Traditionally, the video deployment and session migration is solely executed on the cloud. However, such a simple strategy can hardly guarantee the good trade-off between performance and cost since the popularity of user requests is highly varying, especially for the edge clouds closer to users. In this work, we propose a joint optimization algorithm of session migration and video deployment, i.e., both strategies are smoothly involved to obtain a better trade-off. Consequently, the proposed strategy is more adaptive to dynamic fluctuation of video popularity, and thus gains a flexible balance between service cost and quality. The trace-driven experiment verified the effectiveness of the proposed method, and the results show that our proposed method has better adaptation to popularity than other comparable methods.

In image classification, recognition or retrieval systems, image contents are commonly described by global features. However, the global features generally contain noise from the background, occlusion, or irrelevant objects in the images. Thus, only part of the global feature elements is informative for describing the objects of interest and useful for the image analysis tasks. In this paper, we propose algorithms to automatically discover the subgroups of highly correlated feature elements within predefined global features. To this end, we first propose a novel mixture sparse regression (MSR) method, which groups the elements of a single vector according to the membership conveyed by their sparse regression coefficients. Based on MSR, we proceed to develop the autogrouped sparse representation (ASR), which groups correlated feature elements together through fusing their individual sparse representations over multiple samples. We apply ASR/MSR in two practical visual analysis tasks: 1) multilabel image classification and 2) motion segmentation. Comprehensive experimental evaluations show that our proposed methods are able to achieve superior performance compared with the state-of-the-art classification on these two tasks.

How to generate robust image representations, when there is contamination from noisy pixels within the images, is critical for boosting the performance of image classification methods. However, such an important problem is not fully explored yet. In this paper, we propose a novel image representation learning method, i.e., collaborative linear coding (CLC), to alleviate the negative influence of noisy features in classifying images. Specifically, CLC exploits the correlation among local features in the coding procedure, in order to suppress the interference of noisy features via weakening their responses on coding basis. CLC implicitly divides the extracted local features into different feature subsets, and such feature allocation is indicated by the introduced latent variables. Within each subset, the features are ensured to be highly correlated, and the produced codes for them are encouraged to activate on the identical basis. Through incorporating such regularization in the coding model, the responses of noisy local features are dominated by the responses of informative features due to their rarity compared with the informative features. Thus the final image representation is more robust and distinctive for following classification, compared with the coding methods without considering such high order correlation. Though CLC involves a set of complicated optimization problems, we investigate the special structure of the problems and then propose an efficient alternative optimization algorithm. We verified the effectiveness and robustness of the proposed CLC on multiple image classification benchmark datasets, including Scene 15, Indoor 67, Flower 102, Pet 37, and PASCAL VOC 2011. Compared with the well established baseline LLC, CLC consistently enhances the classification accuracy, especially for the images containing more noises.

Single junction crystalline silicon (c-Si) solar cells featuring a conventionally doped interdigitated back contact heterojunction (IBC-SHJ) structure has approached a record efficiency of 26.6%, which is very close to the practical limit. However, integrating the interdigital p- and n-type amorphous silicon (a-Si:H) layers on the rear surface of Si substrate is of such complexity, posing problem of heavy dependences on expensive manufacturing techniques including plasma-enhanced chemical vapor deposition and photolithography. Its commercial potential is thus always being questioned, and to seek an alternative fabrication procedure, which adapts to cost-effective deposition parallel with simple patterning characteristics, has been a primary research target of related subjects. Here, we demonstrated 20.1% efficiency dopant-free IBC-SHJ solar cells by combining evaporated carrier-selective materials (MoOx and LiFx) and two-steps hard masks alignments, delivering substantial simplifications in the architecture and fabrication procedures. We investigated the effect of intrinsic a-Si:H films with different thicknesses on the passivation and contact resistance for both a-Si:H/MoOx and a-Si:H/LiFx contacts, showing 4 nm a-Si:H is better for high efficiency IBC-SHJ solar cells. We also found that the position of the metal target (electrode definition step) and isolation in between the busbar and the Si substrate are highly relevant to leakage and recombination and have great impact on the device performance. The dopant-free IBC-SHJ solar cells demonstrated here manifest enough confidence in our hard mask based fabrication procedure, with great potential for high performance-to-cost ratio in future.

Research on photovoltaic devices with a high performance-to-cost ratio requires efforts not only on efficiency improvement but also on manufacturing cost reduction. Recently, a record efficiency of 26.6% on crystalline silicon solar cells (SCs) has been achieved by combining the heterojunctions (HJs) with a device structure of interdigitated back contacts. However, the technology that integrates the interdigital p- and n-type amorphous silicon (a-Si:H) layers on the rear surface of the Si substrate is challenging. This issue has motivated researchers to search dopant-free carrier-selective contacts with alternative materials to completely replace doped a-Si:H layers. Transition metal oxides, graphene, and poly(3,4-ethylene dioxythiophene):poly(styrenesulfonate) (PEDOT:PSS), all having high work functions and hole conducting properties, can play the role of hole-selective layers (HSLs). In this review, we focus on the latest advances and the future trends in these HSLs and their applications in silicon HJ SCs. The main issues and challenges encountered are discussed.

PEDOT PSS/silicon heterojunction solar cell has recently attracted much attention due to the fact that it can be simply and cost-effectively fabricated. It is crucial to suppress the interfacial recombination rate between silicon (Si) and organic film for improving device efficiency. In this study, we demonstrated a thickness-dependent passivation effect, i.e. the passivation quality over Si substrate was promoted dramatically with increasing the thickness of PEDOT:PSS layer. The effective minority carrier lifetime increased from 32 μs for 50 nm to 360 μs for 200 nm, which corresponds to a change in implied open circuit voltage (V oc-implied) from 545 to 635 mV. Back-junction hybrid solar cells featuring PEDOT:PSS films at rear side were designed to enable adoption of thick PEDOT:PSS layers without having to worry about parasitic absorption, showing a power conversion efficiency (PCE) of 16.3%. Combined with a proper pre-condition on the Si substrate, the back-junction hybrid solar cell with 200 nm PEDOT:PSS layer received an enhanced PCE of 16.8%. In addition, the improved long-term stability for the back-junction device was also observed. The PCE remained 90% (unsealed) after being stored in ambient atmosphere for 30 days and over 80% (sealed) after 150 days.

In this paper, we propose an object-aware holistic superpixel selection (HPS) method to automatically select the discriminative superpixels of an image for image classification purpose. Through only considering the selected superpixels, the interference of cluttered background on the object can be alleviated effectively and thus the classification performance is significantly enhanced. In particular, for an image, HPS first selects the discriminative superpixels for the characteristics of certain class, which can together match the object template of this class well. In addition, these superpixels compose a class-specific matching region. Through performing such superpixel selection for several most probable classes, respectively, HPS generates multiple class-specific matching regions for a single image. Then, HPS merges these matching regions into an integral object region through exploiting their pixel-level intersection information. Finally, such object region instead of the original image is used for image classification. An appealing advantage of HPS is the ability to alleviate the interference of cluttered background yet not require the object to be segmented out accurately. We evaluate the proposed HPS on four challenging image classification benchmark datasets: Oxford-IIIT PET 37, Caltech-UCSD Birds 200, Caltech 101, and PASCAL VOC 2011. The experimental results consistently show that the proposed HPS can remarkably improve the classification performance.