Search

3D object recognition is one of the most important tasks in 3D data processing, and has been extensively studied recently. Researchers have proposed various 3D recognition methods based on deep learning, among which a class of view-based approaches is a typical one. However, in the view-based methods, the commonly used view pooling layer to fuse multi-view features causes a loss of visual information. To alleviate this problem, in this paper, we construct a novel layer called Dynamic Routing Layer (DRL) by modifying the dynamic routing algorithm of capsule network, to more effectively fuse the features of each view. Concretely, in DRL, we use rearrangement and affine transformation to convert features, then leverage the modified dynamic routing algorithm to adaptively choose the converted features, instead of ignoring all but the most active feature in view pooling layer. We also illustrate that the view pooling layer is a special case of our DRL. In addition, based on DRL, we further present a Dynamic Routing Convolutional Neural Network (DRCNN) for multi-view 3D object recognition. Our experiments on three 3D benchmark datasets show that our proposed DRCNN outperforms many state-of-the-arts, which demonstrates the efficacy of our method.

In this paper, we propose a novel network block, dubbed as second-order spectral transform block, for 3D shape retrieval and classification. This network block generalizes the second-order pooling to 3D surface by designing a learnable non-linear transform on the spectrum of the pooled descriptor. The proposed block consists of following two components. First, the second-order average (SO-Avr) and max-pooling (SO-Max) operations are designed on 3D surface to aggregate local descriptors, which are shown to be more discriminative than the popular average-pooling or max-pooling. Second, a learnable spectral transform parameterized by mixture of power function is proposed to perform non-linear feature mapping in the space of pooled descriptors, i.e., manifold of symmetric positive definite matrix for SO-Avr, and space of symmetric matrix for SO-Max. The proposed block can be plugged into existing network architectures to aggregate local shape descriptors for boosting their performance. We apply it to a shallow network for non-rigid 3D shape analysis and to existing networks for rigid shape analysis, where it improves the first-tier retrieval accuracy by 7.2% on SHREC’14 Real dataset and achieves state-of-the-art classification accuracy on ModelNet40. As an extension, we apply our block to 2D image classification, showing its superiority compared with traditional second-order pooling methods. We also provide theoretical and experimental analysis on stability of the proposed second-order spectral transform block.

Shape classification and segmentation of point cloud data are two of the most demanding tasks in photogrammetry and remote sensing applications, which aim to recognize object categories or point labels. Point convolution is an essential operation when designing a network on point clouds for these tasks, which helps to explore 3D local points for feature learning. In this paper, we propose a novel point convolution (PSConv) using separable weights learned with polynomials for 3D point cloud analysis. Specifically, we generalize the traditional convolution defined on the regular data to a 3D point cloud by learning the point convolution kernels based on the polynomials of transformed local point coordinates. We further propose a separable assumption on the convolution kernels to reduce the parameter size and computational cost for our point convolution. Using this novel point convolution, a hierarchical network (PSNet) defined on the point cloud is proposed for 3D shape analysis tasks such as 3D shape classification and segmentation. Experiments are conducted on standard datasets, including synthetic and real scanned ones, and our PSNet achieves state-of-the-art accuracies for shape classification, as well as competitive results for shape segmentation compared with previous methods.

In this paper we propose a rotation-invariant deep network for point clouds analysis. Point-based deep networks are commonly designed to recognize roughly aligned 3D shapes based on point coordinates, but suffer from performance drops with shape rotations. Some geometric features, e.g., distances and angles of points as inputs of network, are rotation-invariant but lose positional information of points. In this work, we propose a novel deep network for point clouds by incorporating positional information of points as inputs while yielding rotation-invariance. The network is hierarchical and relies on two modules: a positional feature embedding block and a relational feature embedding block. Both modules and the whole network are proven to be rotation-invariant when processing point clouds as input. Experiments show state-of-the-art classification and segmentation performances on benchmark datasets, and ablation studies demonstrate effectiveness of the network design .

To reconstruct surface from unorganized points in three-dimensional Euclidean space, we propose a novel efficient and fast method by using l0 gradient minimization, which can directly measure the sparsity of a solution and produce sharper surfaces. Therefore, the proposed method is particularly effective for sharpening major edges and removing noise. Unlike the Poisson surface reconstruction approach and its extensions, our method does not depend on the accurate directions of normal vectors of the unorganized points. The resulting algorithm is developed using a half-quadratic splitting method and is based on decoupled iterations that are alternating over a smoothing step realized by a Poisson approach and an edge-preserving step through an optimization formulation. This iterative algorithm is easy to implement. Various tests are presented to demonstrate that our method is robust to point noise, normal noise and data holes, and thus produces good surface reconstruction results.

Noble metal nanoparticles strongly adhered to dielectric matrices have been extensively studied because of their potential applications in plasmonic devices based on tunable localized surface plasmon (LSP) excitation. Compared with conventional synthesis methods, the noble metal nanoparticles formed by ion-beam irradiation draw significant interest in recent years because a single layer dispersion of nanoparticles strongly bonded on the dielectric substrate can be obtained. In this paper, important phenomena related to ion-beam surface nanostructuring including ion-induced reshaping of metal nanoparticles, ion-induced core-satellite structure formation, and ion-induced burrowing of these nanoparticles are discussed, with their individual effects on LSP excitation. Consequently, ion-induced surface nanostructuring of Ag-Au bimetallic films on amorphous silica glass and sapphire with tunable LSP excitation are presented. In addition, theoretical studies of far-field and near-field optical properties of these nanoparticles under ion irradiation are introduced, and the enhanced localized electric field (hot spot) is interpreted. Finally, the futures and challenges of the emerging plasmonic applications based on tunable LSP excitations in bio-sensing and surface enhanced Raman spectroscopy (SERS) are presented.

Designing a network on 3D surface for non-rigid shape analysis is a challenging task. In this work, we propose a novel spectral transform network on 3D surface to learn shape descriptors. The proposed network architecture consists of four stages: raw descriptor extraction, surface second-order pooling, mixture of power function-based spectral transform, and metric learning. The proposed network is simple and shallow. Quantitative experiments on challenging benchmarks show its effectiveness for non-rigid shape retrieval and classification, e.g., it achieved the highest accuracies on SHREC’14, 15 datasets as well as the “range” subset of SHREC’17 dataset.

Herein is reported the dots alignment of Au nano-submicron particles (Au-NPs) on a chemical vapor-deposited β-SiC substrate through nanosecond-pulsed laser irradiation-induced dewetting of an Au thin film, which was successfully observed by in-situ high-voltage electron microscopy. The initial stages of the Au-NPs array arrangement were also successfully observed, and long-range quasi-two-dimensional Au-NPs arrangements were found after laser irradiation. Moreover, the effect of Rayleigh instability to form multiple Au-NPs in each line of the network structure after the hole growth in the Au thin film was confirmed.

The in situ observation of Au dot formation and the self-assembly dynamics of Au nanoparticles (NPs) was successfully demonstrated via dewetting of Au thin films on SiO2 glass substrates under nano-second pulsed laser irradiation using a multi-quantum beam high-voltage electron microscope. Moreover, using electron energy-loss spectroscopy (EELS) performed in a scanning transmission electron microscope (STEM), the plasmonic properties of the formed Au/SiO2 nanostructure were analyzed to demonstrate its validity in advanced optical devices. The uniformly distributed Au NPs evolved into a dot alignment through movement and coalescence processes was demonstrated in this in situ observation. We carried out the plasmon-loss images of the plan view and the cross-section of the Au/SiO2 nanostructures were obtained at the plasmon-loss peak energy for investigate the three-dimensional distribution of surface plasmon. Furthermore, discrete-dipole approximation (DDA) calculations were used to simulate the plasmonic properties, such as the surface plasmon resonance and the surface plasmon field distribution, of isolated single Au/SiO2 nanostructures. This STEM-EELS-acquired surface plasmon map of the cross-sectional sample is in excellent agreement with the DDA calculations. This results demonstrated the influence of the contact condition between Au NP and SiO2 glass on the plasmonic properties, and may improve the technology for developing advanced optical devices.