Search

Point cloud accumulation is a crucial technique in point cloud analysis, facilitating various downstream tasks like surface reconstruction. Current methods merely rely on raw LiDAR points, yielding unsatisfactory performance due to the limited geometric information, particularly in complex scenarios characterized by intricate motions, diverse objects, and an increased number of frames. In this paper, we introduce camera modality data, which is usually acquired alongside LiDAR data at minimal expense. To this end, we present the M ultimodal S patiotemporal A ggregation solution (termed MSA) to thoroughly explore and aggregate these two distinct modalities (sparse 3D points and multi-view 2D images). Concretely, we propose a multimodal spatial aggregation module to bridge the data gap between different modalities in the Bird's-Eye-View (BEV) space and further fuse them by learnable adaptive channel-wise weights. By assembling their respective strengths, this module generates a reliable and consistent scene representation. Subsequently, we design a temporal aggregation module to capture continuous motion information across consecutive sequences, which is beneficial for identifying the motion state of the foreground scene and enabling the model to extend robustly to longer sequences. Experiments demonstrate MSA outperforms state-of-the-art (SoTA) point cloud accumulation methods across all evaluation metrics in the public benchmark, especially with more frames. [ABSTRACT FROM AUTHOR]

[Display omitted] • Magnetic particles integrated LC-MS strategy guides isolation of trace compounds. • Fifteen trace alkaloids including seven new ones were obtained from Nuphar pumila. • The Δ δ C of methylenes at sulfur-ring concludes the stereochemistry of spiroatoms. • The immunosuppressive effects of Nuphar dimeric alkaloids were revealed. Trace natural products (TNPs) are still the vital source of drug development. However, the mining of novel TNPs is becoming increasingly challenging due to their low abundance and complex interference. A comprehensive strategy was proposed in which the functionalized magnetic particles integrated with LC-MS for TNPs discovery. Under the guidance of the approach, fifteen trace Nuphar alkaloids including seven new ones, cyanopumiline A sulfoxide (1), cyanopumiline C sulfoxide (8) and cyanopumilines A–E (4 – 5 , 10 , 12 – 13) featuring an undescribed nitrile-containing 6/6/5/6/6 pentacyclic ring system were isolated from the rhizomes of Nuphar pumila. Their structures and absolute configurations were determined on the basis of detailed spectroscopic data analysis and single-crystal X-ray diffraction analysis. Notably, a concise method based on 13C NMR spectroscopy was established to determine the relative configurations of spiroatoms. Biologically, compounds 1 – 12 exhibited potent immunosuppressive activities with IC 50 values ranging from 0.1–12.1 μM against anti-CD3/CD28 induced human peripheral T cell proliferation. Mechanistic studies revealed that 4 could dose-dependently decrease pro-inflammatory cytokines and the expression levels of CD25 and CD71. [ABSTRACT FROM AUTHOR]

Unlike 2D saliency detection, 3D saliency detection can consider the effects of depth and binocular parallax. In this paper, we propose a 3D saliency detection approach based on background detection via depth information. With the aid of the synergism between a color image and the corresponding depth map, our approach can detect the distant background and surfaces with gradual changes in depth. We then use the detected background to predict the potential characteristics of the background regions that are occluded by foreground objects through polynomial fitting; this step imitates the human imagination/envisioning process. Finally, a saliency map is obtained based on the contrast between the foreground objects and the potential background. We compare our approach with 14 state-of-the-art saliency detection methods on three publicly available databases. The proposed model demonstrates good performance and succeeds in detecting and removing backgrounds and surfaces of gradually varying depth on all tested databases. [ABSTRACT FROM AUTHOR]

Object detection on 360°images is a vital component of 3D environment perception. The existing methods either treat panoramic images (usually represented in equirectangular projection—ERP) as normal FoV images and endure the distortions or project them into the less-distortion format and narrow the FoV, leading to unsatisfactory performance in practical applications. To solve this problem, we propose a dual-projection 360°object detection network named Bip R-CNN , consisting of three modules: a bi-projection feature extractor, a cross-projection region-of-interest (RoI) searcher, and a classification and regression predictor. Specifically, we extract the equirectangular and corresponding dual-cubemap features simultaneously from the input images. Besides, Projection-Inter Feature Fusion and Projection-Intra Feature Fusion are designed to allow the mutual interaction between the bi-projective features and promote the integration of features at different scales, respectively. In the proposed cross-projection RoI Searcher, we search for the bounding box (BBox) locations on cubemap from the corresponding ERP spherical proposals, bridging the RoIs of two different projection formats at feature level. Finally, the cube proposals are used to detect objects in the last predictor module. Considering the scarceness of the existing panoramic dataset (only indoor scenes), we propose an efficient approach to convert conventional datasets into annotated panoramic datasets without manual intervention, increasing the diversity of panoramic datasets. Extensive experiments are conducted on the synthetic and real-world datasets with spherical criteria, demonstrating our superiority to other state-of-the-art solutions. [ABSTRACT FROM AUTHOR]

Image stitching is a traditional but challenging computer vision task, aiming to obtain a seamless panoramic image. Recently, researchers begin to study the image stitching task using deep learning. However, the existing learning methods assume a relatively fixed view during the image capturing, thus show a poor generalization ability to flexible view cases. To address the above problem, we present a cascaded view-free image stitching network based on a global homography. This novel image stitching network does not have any restriction on the view of images and it can be implemented in three stages. In particular, we first estimate a global homography between two input images from different views. And then we propose a structure stitching layer to obtain the coarse stitching result using the global homography. In the last stage, we design a content revision network to eliminate ghosting effects and refine the content of the stitching result. To enable efficient learning on various views, we also present a method to generate synthetic datasets for network training. Experimental results demonstrate that our method can achieve almost 100% elimination of artifacts in overlapping areas at the cost of acceptable slight distortions in non-overlapping areas, compared with traditional methods. In addition, the proposed method is view-free and more robust especially in a scene where feature points are difficult to detect. [ABSTRACT FROM AUTHOR]

Neural network based methods for fisheye distortion correction are effective and increasingly popular, although training network require a high amount of labeled data. In this paper, we propose an unsupervised fisheye correction network to address the aforementioned issue. During the training process, the predicted parameters are employed to correct strong distortion that exists in the fisheye image and synthesize the corresponding distortion using the original distortion-free image. Thus, the network is constrained with bidirectional loss to obtain more accurate distortion parameters. We calculate the two losses at the image level as opposed to directly minimizing the difference between the predicted and ground truth of distortion parameters. Additionally, we leverage the geometric prior that the distortion distribution depends on the geometric regions of fisheye images and the straight line should be straight in the corrected images. The network focuses more on the geometric prior regions as opposed to equally perceiving the whole image without any attention mechanisms. To generate more appealing corrected results in visual appearance, we introduce a coarse-to-fine inpainting network to fill the hole regions caused by the irreversible mapping function using distortion parameters. Each module of the proposed network is differentiable, and thus the entire framework is completely end-to-end. When compared with the previous supervised methods, our method is more flexible and shows better practical applications for distortion rectification. The experiment results demonstrate that our proposed method outperforms state-of-the-art methods on the correction performance without any labeled distortion parameters. [ABSTRACT FROM AUTHOR]