Search

LiDAR sensors are widely used in autonomous driving due to the reliable 3D spatial information. However, the data of LiDAR is sparse and the frequency of LiDAR is lower than that of cameras. To generate denser point clouds spatially and temporally, we propose the first future pseudo-LiDAR frame prediction network. Given the consecutive sparse depth maps and RGB images, we first predict a future dense depth map based on dynamic motion information coarsely. To eliminate the errors of optical flow estimation, an inter-frame aggregation module is proposed to fuse the warped depth maps with adaptive weights. Then, we refine the predicted dense depth map using static contextual information. The future pseudo-LiDAR frame can be obtained by converting the predicted dense depth map into corresponding 3D point clouds. Extensive experiments are conducted in-depth completion, pseudo-LiDAR interpolation, and LiDAR prediction. Our approach achieves state-of-the-art performance on all the above tests on the popular KITTI dataset, and the primary evaluation metric RMSE reaches 1214. [ABSTRACT FROM AUTHOR]

Homography estimation is an important task in computer vision applications, such as image stitching, video stabilization, and camera calibration. Traditional homography estimation methods heavily depend on the quantity and distribution of feature correspondences, leading to poor robustness in low-texture scenes. The learning solutions, on the contrary, try to learn robust deep features but demonstrate unsatisfying performance in the scenes with low overlap rates. In this paper, we address these two problems simultaneously by designing a contextual correlation layer (CCL). The CCL can efficiently capture the long-range correlation within feature maps and can be flexibly used in a learning framework. In addition, considering that a single homography can not represent the complex spatial transformation in depth-varying images with parallax, we propose to predict multi-grid homography from global to local. Moreover, we equip our network with a depth perception capability, by introducing a novel depth-aware shape-preserved loss. Extensive experiments demonstrate the superiority of our method over state-of-the-art solutions in the synthetic benchmark dataset and real-world dataset. The codes and models will be available at https://github.com/nie-lang/Multi-Grid-Deep-Homography. [ABSTRACT FROM AUTHOR]

Image stitching is a classical and challenging technique in computer vision, which aims to generate an image with a wide field of view. The traditional methods heavily depend on feature detection and require the feature points to be dense and evenly distributed in the image, leading to poor robustness in low-texture scenes. Learning methods are rarely studied due to the unavailability of ground truth stitched results, showing unreliable performance on real-world datasets. In this paper, we propose an image stitching learning framework, which consists of a multi-scale deep homography module and an edge-preserved deformation module. First, we design a multi-scale deep homography module to estimate the accurate homography progressively from coarse to fine. After that, an edge-preserved deformation module is designed to learn the deformation rules of image stitching from edge to content, generating the stitched image with artifacts eliminated. Besides, the proposed supervised learning framework can stitch images of arbitrary resolutions and demonstrate good generalization capability in real-world images. Experiments show that our superiority to the existing homography solutions and image stitching algorithms. [ABSTRACT FROM AUTHOR]

Fisheye cameras can capture a large field-of-view (Fov) scene but it introduces severe radial distortion in images. Thus, distortion rectification is a crucial step for subsequent computer vision tasks using fisheye cameras. A prevalent type of method predicts the displacement field between the input and output to rectify the distorted images. However, it is challenging to estimate the accurate flow in Cartesian coordinates (both $x$ and $y$ directions), in which the sampling strategy of the convolution kernel ignores the radial symmetry of distortion. In general, the pixel’s distortion at the same radius from the center is the same, while the radius corresponds to one parameter in polar coordinates. Motivated by this fact, we exploit the radial symmetry of distortion to predict a more straightforward one-dimensional flow, transforming the distorted image into the polar coordinates domain instead of predicting two-dimensional flow in $x$ and $y$ directions. Specifically, we propose a Polar coordinates Distortion Rectification Network (PCDRN), whose sampling strategy corresponds to the radial distortion characteristic so that a more accurate flow can be predicted. To eliminate the blurs and ring artifacts induced by the coordinates transformation, a Polar-To-Cartesian Appearance Enhancement Network is designed to enhance the local appearance of rectified images. Experimental results on the synthesized dataset and real-world dataset demonstrate the superiority of our approach in both quantitative and qualitative evaluations. [ABSTRACT FROM AUTHOR]

[Display omitted] • Magnetic particles integrated LC-MS strategy guides isolation of trace compounds. • Fifteen trace alkaloids including seven new ones were obtained from Nuphar pumila. • The Δ δ C of methylenes at sulfur-ring concludes the stereochemistry of spiroatoms. • The immunosuppressive effects of Nuphar dimeric alkaloids were revealed. Trace natural products (TNPs) are still the vital source of drug development. However, the mining of novel TNPs is becoming increasingly challenging due to their low abundance and complex interference. A comprehensive strategy was proposed in which the functionalized magnetic particles integrated with LC-MS for TNPs discovery. Under the guidance of the approach, fifteen trace Nuphar alkaloids including seven new ones, cyanopumiline A sulfoxide (1), cyanopumiline C sulfoxide (8) and cyanopumilines A–E (4 – 5 , 10 , 12 – 13) featuring an undescribed nitrile-containing 6/6/5/6/6 pentacyclic ring system were isolated from the rhizomes of Nuphar pumila. Their structures and absolute configurations were determined on the basis of detailed spectroscopic data analysis and single-crystal X-ray diffraction analysis. Notably, a concise method based on 13C NMR spectroscopy was established to determine the relative configurations of spiroatoms. Biologically, compounds 1 – 12 exhibited potent immunosuppressive activities with IC 50 values ranging from 0.1–12.1 μM against anti-CD3/CD28 induced human peripheral T cell proliferation. Mechanistic studies revealed that 4 could dose-dependently decrease pro-inflammatory cytokines and the expression levels of CD25 and CD71. [ABSTRACT FROM AUTHOR]

Surround-view system (SVS) is widely used in the advanced driver assistance system (ADAS). SVS uses four fish-eye lenses to monitor real-time scenes around the vehicle. However, accurate intrinsic and extrinsic parameter estimation is required for the proper functioning of the system. At present, the intrinsic calibration can be pipeline by utilizing checkerboard algorithm, while extrinsic calibration is still immature. Therefore, we proposed a specific calibration pipeline to estimate extrinsic parameters robustly. This scheme takes a driving sequence of four cameras as input. It firstly utilizes lane line to roughly estimate each camera pose. Considering the environmental condition differences in each camera, we separately select strategies from two methods to accurately estimate the extrinsic parameters. To achieve accurate estimates for both front and rear camera, we proposed a method that mutually iterating line detection and pose estimation. As for bilateral camera, we iteratively adjust the camera pose and position by minimizing texture and edge error between ground projections of adjacent cameras. After estimating the extrinsic parameters, the surround-view image can be synthesized by homography-based transformation. The proposed pipeline can robustly estimate the four SVS camera extrinsic parameters in real driving environments. In addition, to evaluate the proposed scheme, we build a surround-view fish-eye dataset, which contains 40 videos with 32,000 frames, acquired from different real traffic scenarios. All the frames in each video are manually labeled with lane annotation, with its GT extrinsic parameters. Moreover, this surround-view dataset could be used by other researchers to evaluate their performance. The dataset will be available soon. [ABSTRACT FROM AUTHOR]

Traditional feature-based image stitching technologies rely heavily on feature detection quality, often failing to stitch images with few features or low resolution. The learning-based image stitching solutions are rarely studied due to the lack of labeled data, making the supervised methods unreliable. To address the above limitations, we propose an unsupervised deep image stitching framework consisting of two stages: unsupervised coarse image alignment and unsupervised image reconstruction. In the first stage, we design an ablation-based loss to constrain an unsupervised homography network, which is more suitable for large-baseline scenes. Moreover, a transformer layer is introduced to warp the input images in the stitching-domain space. In the second stage, motivated by the insight that the misalignments in pixel-level can be eliminated to a certain extent in feature-level, we design an unsupervised image reconstruction network to eliminate the artifacts from features to pixels. Specifically, the reconstruction network can be implemented by a low-resolution deformation branch and a high-resolution refined branch, learning the deformation rules of image stitching and enhancing the resolution simultaneously. To establish an evaluation benchmark and train the learning framework, a comprehensive real-world image dataset for unsupervised deep image stitching is presented and released. Extensive experiments well demonstrate the superiority of our method over other state-of-the-art solutions. Even compared with the supervised solutions, our image stitching quality is still preferred by users. [ABSTRACT FROM AUTHOR]

Radial distortion has widely existed in the images captured by popular wide-angle cameras and fisheye cameras. Despite the long history of distortion rectification, accurately estimating the distortion parameters from a single distorted image is still challenging. The main reason is that these parameters are implicit to image features, influencing the networks to learn the distortion information fully. In this work, we propose a novel distortion rectification approach that can obtain more accurate parameters with higher efficiency. Our key insight is that distortion rectification can be cast as a problem of learning an ordinal distortion from a single distorted image. To solve this problem, we design a local-global associated estimation network that learns the ordinal distortion to approximate the realistic distortion distribution. In contrast to the implicit distortion parameters, the proposed ordinal distortion has a more explicit relationship with image features, and significantly boosts the distortion perception of neural networks. Considering the redundancy of distortion information, our approach only uses a patch of the distorted image for the ordinal distortion estimation, showing promising applications in efficient distortion rectification. In the distortion rectification field, we are the first to unify the heterogeneous distortion parameters into a learning-friendly intermediate representation through ordinal distortion, bridging the gap between image feature and distortion rectification. The experimental results demonstrate that our approach outperforms the state-of-the-art methods by a significant margin, with approximately 23% improvement on the quantitative evaluation while displaying the best performance on visual appearance. [ABSTRACT FROM AUTHOR]

Distortion rectification is a fundamental task in the field of computer vision and image processing. Nevertheless, previous methods have regarded distortion rectification as a static problem that learns a mapping function and corrects the distorted image to a unique state. However, this state is generally not the optimal solution, as it would result in an under-rectified or over-rectified structure. In this study, we revisit the classical distortion rectification task with a new perspective and redesign the algorithm, inspired by video processing techniques. Specifically, we regard distortion rectification as a dynamic problem that can be extended to a sequence of different distortion states: the input distorted image (t), under-rectified image (t+1), ideal-rectified image (t+2), and over-rectified image (t+3). We first estimate the residual distortion map (RDM) between the input distorted image and the coarse-rectified (t+1 or t+3) image. Here, RDM indicates the motion difference between two distorted images. Subsequently, the RDM is used to guide the refinement rectification process, aiming to convert the coarse-rectified state into the ideal-rectified state. In addition, the flexible implementation of the proposed refinement process with RDM to improve the rectification results of any method is appealing. The experimental results demonstrate that our method outperforms the state-of-the-art schemes by a significant margin, revealing approximately 40% improvement through quantitative evaluation. [ABSTRACT FROM AUTHOR]

360° video is very popular due to its 360° views of a scene. Although 360° videos are also compressed by a hybrid coding framework like 2D video, its high resolution and serious shape deformation affect coding efficiency. In equirectangular projection (ERP) format of 360° videos, if an object moves from equator regions to pole regions or vice versa, large deformation will be introduced and motion estimation cannot find the best‐matched part. To solve the above problem, the authors propose to generate a better reference frame for the current to be encoded frame. First, they project the frame prior to the current one from ERP to the sphere and rotate it at an appropriate angle depending on motion vectors. Subsequently, they insert this generated frame to the rear of the reference queue and let the encoder work as usual. The advantage is that the inserted frame has a more similar shape deformation as the current frame, which greatly helps motion estimation and makes full use of 360° video characters. Their method is simple and friendly compatible with the existing compression standard. Experiments prove that their method achieves 1.57% Bjøntegaard Delta (BD)‐gain compared with standard high efficiency video coding. [ABSTRACT FROM AUTHOR]

Recently, learning-based distortion rectification schemes have shown high efficiency. However, most of these methods only focus on a specific camera model with fixed parameters, thus failing to be extended to other models. To avoid such a disadvantage, we propose a model-free distortion rectification framework for the single-shot case, bridged by the distortion distribution map (DDM). Our framework is based on an observation that the pixel-wise distortion information is explicitly regular in a distorted image, despite different models having different types and numbers of distortion parameters. Motivated by this observation, instead of estimating the heterogeneous distortion parameters, we construct a proposed distortion distribution map that intuitively indicates the global distortion features of a distorted image. In addition, we develop a dual-stream feature learning module, benefitting from both the advantages of traditional methods that leverage the local handcrafted feature and learning-based methods that focus on the global semantic feature perception. Due to the sparsity of handcrafted features, we discrete the features into a 2D point map and learn the structure inspired by PointNet. Finally, a multimodal attention fusion module is designed to attentively fuse the local structural and global semantic features, providing the hybrid features for the more reasonable scene recovery. The experimental results demonstrate the excellent generalization ability and more significant performance of our method in both quantitative and qualitative evaluations, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Radial distortion, which severely hinders object detection and semantic recognition, frequently exists in images captured using a wide-angle lens. Correction of this distortion of images is crucial in many computer vision applications. In this paper, we present distortion rectification generative adversarial network (DR-GAN), a conditional generative adversarial network (GAN) for automatic radial DR. To the best of our knowledge, this is the first end-to-end trainable adversarial framework for radial distortion rectification. The DR-GAN trained using the proposed low-to-high perceptual loss learns the mapping relation between different structural images rather than estimating multifarious distortion parameters, while also realizing label-free training and one-stage rectification. As a benefit of one-stage rectification, the proposed method is extremely fast with the completion of rectification in real time. This is approximately 22 times faster than the state-of-the-art methods. The experimental results show that the DR-GAN achieves an excellent performance in both quantitative measure (PSNR and SSIM) and visual qualitative appearance. [ABSTRACT FROM AUTHOR]

Unlike 2D saliency detection, 3D saliency detection can consider the effects of depth and binocular parallax. In this paper, we propose a 3D saliency detection approach based on background detection via depth information. With the aid of the synergism between a color image and the corresponding depth map, our approach can detect the distant background and surfaces with gradual changes in depth. We then use the detected background to predict the potential characteristics of the background regions that are occluded by foreground objects through polynomial fitting; this step imitates the human imagination/envisioning process. Finally, a saliency map is obtained based on the contrast between the foreground objects and the potential background. We compare our approach with 14 state-of-the-art saliency detection methods on three publicly available databases. The proposed model demonstrates good performance and succeeds in detecting and removing backgrounds and surfaces of gradually varying depth on all tested databases. [ABSTRACT FROM AUTHOR]

In texture-plus-depth format of three-dimensional visual data, both texture and depth maps are required to synthesize a desired view via depth-image-based rendering. However, the depth maps captured or estimated always exist with low resolution compared to their corresponding texture images. We introduce a joint edge-guided convolutional neural network that upsamples the resolution of a depth map on the premise of synthesized view quality. The network takes the low-resolution depth map as an input using a joint edge feature extracted from the depth map and the registered texture image as a reference, and then produces a highresolution depth map. We further use local constraints that preserve smooth regions and sharp edges so as to improve the quality of the depth map and synthesized view. Finally, a global looping optimization is performed with virtual view quality as guidance in the recovery process. We train our model using pairs of depth maps and texture images and then make tests on other depth maps and video sequences. The experimental results demonstrate that our scheme outperforms existing methods both in the quality of the depth maps and synthesized views. [ABSTRACT FROM AUTHOR]

Twelve undescribed compounds including five sesquiterpenes (1 – 5), one monoterpene (6), and four lignans (7a / 7b and 8a / 8b), along with two other types (9 and 10) were isolated from the rhizomes of Atractylodes macrocephala. Among them, two pairs of enantiomers (7a / 7b and 8a / 8b) were successfully separated by chiral-phase HPLC, while racemate 9 could not be resolved. Their structures and absolute configurations were unambiguously elucidated by spectroscopic data analysis and electronic circular dichroism (ECD) calculations. Notably, compounds 1 and 2 are rare sesquiterpene hybrids featuring an eudesmanolactam linked to a resorcinol or methyl 2-methylpentanoat through a C N bond. Compound 3 represents the first example of eudesmanolide sesquiterpene with an oxygen-bridge between C-8 and C-14. Compounds 7a and 7b are a pair of rare enantiomeric benzodioxane norneolignans. Additionally, compound 2 exhibited weak cytotoxicity against SGC-7901 cells. Compound 4 significantly promoted the proliferation of LPS-induced IEC-6 cells with the rate of 117.2%. [Display omitted] • Four types of undescribed compounds were obtained from A. macrocephala. • Atramacronoids S and T are rare hybrids of N -containing sesquiterpenes and others. • A pair of rare enantiomeric norneolignans were separated by chiral column. • Atramacronoid V had the capacity of promoting IEC-6 cells proliferation. [ABSTRACT FROM AUTHOR]

Object detection on 360°images is a vital component of 3D environment perception. The existing methods either treat panoramic images (usually represented in equirectangular projection—ERP) as normal FoV images and endure the distortions or project them into the less-distortion format and narrow the FoV, leading to unsatisfactory performance in practical applications. To solve this problem, we propose a dual-projection 360°object detection network named Bip R-CNN , consisting of three modules: a bi-projection feature extractor, a cross-projection region-of-interest (RoI) searcher, and a classification and regression predictor. Specifically, we extract the equirectangular and corresponding dual-cubemap features simultaneously from the input images. Besides, Projection-Inter Feature Fusion and Projection-Intra Feature Fusion are designed to allow the mutual interaction between the bi-projective features and promote the integration of features at different scales, respectively. In the proposed cross-projection RoI Searcher, we search for the bounding box (BBox) locations on cubemap from the corresponding ERP spherical proposals, bridging the RoIs of two different projection formats at feature level. Finally, the cube proposals are used to detect objects in the last predictor module. Considering the scarceness of the existing panoramic dataset (only indoor scenes), we propose an efficient approach to convert conventional datasets into annotated panoramic datasets without manual intervention, increasing the diversity of panoramic datasets. Extensive experiments are conducted on the synthetic and real-world datasets with spherical criteria, demonstrating our superiority to other state-of-the-art solutions. [ABSTRACT FROM AUTHOR]

• The effect of nozzle size, CO 2 and H 2 O molar fraction, pressure, and temperature on both the infrared characteristics and similarity are studied. • It is found that the essential conditions for the radiation similarity criterion are identical temperature distributions and identical characteristic optical thicknesses of the jet flow. • The spectral radiance increases at 3 ∼ 3.6 μm and 4.6 ∼ 5 μm, while it decreases around 4.3 μm when the above parameters increase. In this study, the effects of nozzle size, CO 2 and H 2 O molar fraction, pressure, and temperature on the infrared characteristics of round jets are studied. The backward Monte Carlo method is used to calculate the spectral radiation characteristics and infrared images. The results show that k λ is insensitive to the molar fraction of the species, and it is slightly more sensitive to pressure. Temperature has the greatest effect on the k λ. In addition, the effects of characteristic length of the jet, molar fraction of the species, and pressure on the infrared characteristics are similar. (1) When the characteristic optical thickness of the jet is thick enough, with the increasing of above parameters, the spectral radiance increases at 3 ∼ 3.6 μm and 4.6 ∼ 5 μm, while it decreases around 4.3 μm. The magnitude of the two radiance peaks near 4.1 μm and 4.4 μm remain almost unchanged. However, the peak position moves to both sides. (2) Otherwise, there will be only one radiance peak in 4.1 ∼ 4.4 μm. The height of radiance peak decreases with the decrease of characteristic optical thickness. [ABSTRACT FROM AUTHOR]

Due to the prediction structures employed in video coding, the loss of one packet will affect many following frames. In this paper, a multiple description coding scheme with stagger frame order is proposed for stereoscopic 3-D videos. First, the reference and auxiliary views in stereoscopic sequences will be asymmetrically encoded into one description, whereas the other description will be formed in the same way with one dumb frame delay. Because of the stagger frame order, the coarsely encoded B frames will be inserted into different positions of the two descriptions. If a certain frame encoded with I/P mode is lost, then its corresponding B-frame version will be employed to compensate for the loss. In each description, the quantization steps of B frames are tuned based on a closed-form solution that considers the video contents, network status, frame positions in the group of picture, and the layer of the views. For further improvement, a fusing scheme is provided. The experimental results demonstrate that the proposed scheme outperforms state-of-the-art schemes. Specifically, up to 1.3-dB gain is achieved in the case of packet loss, and 2-dB gain is obtained for the side/central performance. [ABSTRACT FROM PUBLISHER]

In this paper, a novel multiple description video coding scheme is proposed to insert and control the redundancy at macro block (MB) level. By analyzing the error propagation paths, the relative importance of each MB is determined. The paths, in practice, depend on both the video content and the adopted video coder. Considering the relative importance of the MB and the network status, an unequal protection for the video data can be realized to exploit the redundancy effectively. In addition, a simple and effective approach is introduced to tune the quantization parameter for the variable rate coding case. The whole scheme is implemented in H.264/AVC by employing its coding options, thus generating descriptions that are compatible with the baseline profile and extended profile of H.264/AVC. Due to its general property, the proposed approach can be employed for other hybrid video codecs. The results demonstrate the advantage of the proposed approach over other H.264/AVC multiple description schemes. [ABSTRACT FROM AUTHOR]

In this paper, we incorporate Trellis Coded Quantization (TCQ) into a two-stage multiple description coding structure to obtain granular gain over two-stage multiple description Scalar Quantizer (SQ). Analysis and experiment on Gaussian signal show that the performance of the proposed scheme can achieve larger gain than that of the two-stage SQ scheme because of better performance of TCQ. The proposed scheme for image coding is shown to be more effective than other relevant multiple description image coding schemes in terms of central-side-distortion rate performance. [ABSTRACT FROM AUTHOR]

Eight new 11-methoxymethylgermacranolides (1–8) were isolated from the ethanol extract of the whole plant of Carpesium divaricatum. The planar structures and relative configurations of the new compounds were determined by detailed spectroscopic analysis. The absolute configuration of 1 was established by electronic circular dichroism (ECD) spectrum and X-ray crystallographic analysis, and the stereochemistry of the new compounds 2–8 were determined by similar ECD data with 1. The absolute configurations of 5 and 7 were further confirmed by using quantum chemical electronic circular dichroism (ECD) calculations. Compound 4 exhibited weak cytotoxicity against MCF-7 cells. Compound 8 could potently decrease PGE2 productions in LPS-induced RAW 264.7 cells. [ABSTRACT FROM AUTHOR]

Image stitching is a traditional but challenging computer vision task, aiming to obtain a seamless panoramic image. Recently, researchers begin to study the image stitching task using deep learning. However, the existing learning methods assume a relatively fixed view during the image capturing, thus show a poor generalization ability to flexible view cases. To address the above problem, we present a cascaded view-free image stitching network based on a global homography. This novel image stitching network does not have any restriction on the view of images and it can be implemented in three stages. In particular, we first estimate a global homography between two input images from different views. And then we propose a structure stitching layer to obtain the coarse stitching result using the global homography. In the last stage, we design a content revision network to eliminate ghosting effects and refine the content of the stitching result. To enable efficient learning on various views, we also present a method to generate synthetic datasets for network training. Experimental results demonstrate that our method can achieve almost 100% elimination of artifacts in overlapping areas at the cost of acceptable slight distortions in non-overlapping areas, compared with traditional methods. In addition, the proposed method is view-free and more robust especially in a scene where feature points are difficult to detect. [ABSTRACT FROM AUTHOR]

Neural network based methods for fisheye distortion correction are effective and increasingly popular, although training network require a high amount of labeled data. In this paper, we propose an unsupervised fisheye correction network to address the aforementioned issue. During the training process, the predicted parameters are employed to correct strong distortion that exists in the fisheye image and synthesize the corresponding distortion using the original distortion-free image. Thus, the network is constrained with bidirectional loss to obtain more accurate distortion parameters. We calculate the two losses at the image level as opposed to directly minimizing the difference between the predicted and ground truth of distortion parameters. Additionally, we leverage the geometric prior that the distortion distribution depends on the geometric regions of fisheye images and the straight line should be straight in the corrected images. The network focuses more on the geometric prior regions as opposed to equally perceiving the whole image without any attention mechanisms. To generate more appealing corrected results in visual appearance, we introduce a coarse-to-fine inpainting network to fill the hole regions caused by the irreversible mapping function using distortion parameters. Each module of the proposed network is differentiable, and thus the entire framework is completely end-to-end. When compared with the previous supervised methods, our method is more flexible and shows better practical applications for distortion rectification. The experiment results demonstrate that our proposed method outperforms state-of-the-art methods on the correction performance without any labeled distortion parameters. [ABSTRACT FROM AUTHOR]

Fisheye lenses are widely applied in the self-driving system due to its large field of view (FOV), which also induces the strong radial distortion in captured data. Distortion rectification is a crucial preprocessing step for scene perception since the non-linear projection heavily destroys the original geometric distribution of objects. Previous methods rectify the distorted image with a plausible global distribution, but they perform poorly on the local appearance such as the detailed texture of vehicles and pedestrians, which are hard to meet the fine requirements of object detection and semantic recognition. To solve this problem, we propose a joint distortion rectification (DR) and super-resolution (SR) end-to-end framework to generate the realistic rectification result with a visually pleasing local appearance. To the best of our knowledge, this is the first parametric framework that combines the challenging techniques DR and SR in the field of computer vision, which learns the accurate mapping function from the distorted and low-resolution domain into the corrected and high-resolution domain simultaneously. Moreover, the devised network architecture using the channel attention mechanism and object-aware loss function enables better-reconstructed scenes compared with the state-of-the-art methods. Experimental results show that our approach achieves superior rectification and enhancement performance in both quantitative measure and visual qualitative appearance, contributing a comprehensive and promising vision algorithm to the perception of the self-driving scene. [ABSTRACT FROM AUTHOR]

Recently, the pseudo-LiDAR gains increasing attention since it provides the probability of replacing the expensive LiDAR with the camera in autonomous driving. However, due to the fixed field of view of the camera, the pseudo-LiDAR point cloud suffers from the limited spatial range. In this paper, we present a novel pseudo-LiDAR point cloud magnification algorithm, aiming to extrapolate the narrow baseline and further bridge the gap between LiDAR and camera. To achieve this goal, we design a complete pipeline consisting of the hybrid view synthesis module, stereo depth estimation module, image-depth associated stitching module, and magnified pseudo-LiDAR point cloud transformation module. Considering the artifacts existed in generated views, we propose a region-aware restoration approach to obtain more realistic synthesized results. To the best of our knowledge, this is the first work for the pseudo-LiDAR point cloud magnification, which shows appealing and meaningful applications in 3D spatial perception systems only equipped with cameras. Experimental results on KITTI benchmark demonstrate that our algorithm can effectively magnify pseudo-LiDAR point cloud with a wider field of view, note that we only use two images captured by stereo cameras. [ABSTRACT FROM AUTHOR]

Abstract Human protein tyrosine phosphatase beta (HPTPβ) inhibitors have been used for the treatment of sepsis, cancer and inflammatory diseases. The phenylsulfamic acid derivatives are a group of powerful inhibitors of HPTPβ. To explore the relationship between their structures and biological activities, the three-dimensional quantitative structure-activity relationship (3D-QSAR) model is first constructed. Two highly predictive 3D-QSAR models are established. These models are the comparative molecular field analysis (CoMFA: q cv 2 0.611, R ncv 2 0.999) model and the comparative molecular similarity index analysis (CoMSIA: q cv 2 0.588, R ncv 2 0.993) model. The results show a quite good external predictive power for the test set, with R pre 2 values of 0.833 and 0.775, respectively. Furthermore, the contour maps of the 3D-QSAR models are analysed together with the results of molecular docking. The analysed results are conducive to discovering new binding sites and provide a reference for the construction of new potent HPTPβ inhibitors compounds. Graphical abstract Image 1 Highlights • Phenylsulfamic acid derivatives, HPTPβ inhibitors, are collected, optimized and calculated to establish the 3D-QSAR model. • Good internal and external predictive power for the training and test sets (q2 cv : 0.611, respectively, 0.588 and R pred 2: 0.833, 0.775, respectively). • The QSAR models provide us some tips for the construction of new potent HPTPβ inhibitors. [ABSTRACT FROM AUTHOR]

Image denoising which aims to restore a high-quality image from the noisy version is one of the most challenging tasks in the low-level computer vision tasks. In this paper, we propose a multi-stage progressive denoising network (MSPNet) and decompose the denoising task into some sub-tasks to progressively remove noise. Specifically, MSPNet is composed of three denoising stages. Each stage combines a feature extraction module (FEM) and a mutual-learning fusion module (MFM). In the feature extraction module, an encoder-decoder architecture is employed to learn non-local contextualized features, and the channel attention blocks (CAB) are utilized to retain the local information of the image. In the mutual-learning fusion module, the criss-cross attention is introduced to balance the image spatial details and the contextualized information. Compared with the state-of-the-art works, experimental results show that MSPNet achieves notable improvements on both objective and subjective evaluations. [ABSTRACT FROM AUTHOR]

For real-time video streaming systems, the video packets arriving after the display deadline of their frames are considered as late-arrival packets, and typically they are discarded. This will affect the current frame and the following ones due to error propagations. For this reason, in this paper, we propose an approach to exploit the late-arrival and out-of-order packets, which includes two mechanisms. The first mechanism will use these packets to update the reference frames to make them more consistent with the encoder side, and this will eventually reduce the error propagations. The second mechanism will use these packets to increase the chance of successfully decoding the Reed-Solomon (RS) code. In the proposed approach, a sub-GOP based systematic RS code is used and optimized to exploit these packets, where the size of each sub-GOP and the parity packet number for each sub-GOP are optimally tuned, taking into consideration the maximum end-to-end delay, the network conditions, and other system parameters, so as to make the best use of the late-arrival packets and to exploit the out-of-order packets. Finally, the experimental results show the advantage of the proposed approach over other approaches. [ABSTRACT FROM AUTHOR]

Ultra-high definition (UHD) 360 videos encoded in fine quality are typically too large to stream in its entirety over bandwidth (BW)-constrained networks. One popular approach is to interactively extract and send a spatial sub-region corresponding to a viewer’s current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction foretelling a viewer’s future FoVs is essential. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem: three sources of relevant information—collected viewers’ head movement traces, a 360 image saliency map, and a biological human head model—are distilled into a view transition Markov model. Specifically, we formulate a constrained maximum a posteriori (MAP) problem with likelihood and prior terms defined using the three information sources. We solve the MAP problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe (FW) optimization strategy. In each FW iteration, a linear program (LP) is solved, whose runtime is reduced thanks to warm start initialization. Having estimated a Markov model from data, we employ it to optimize a tile-based 360 video streaming system. Extensive experiments show that our head movement prediction scheme noticeably outperformed existing proposals, and our optimized tile-based streaming scheme outperformed competitors in rate-distortion performance. [ABSTRACT FROM AUTHOR]

The DNA repair enzyme 8‐oxoguanine DNA glycosylase‐1 (OGG1) is involved in early embryonic development, as well as in multiple conditions, including cardiac fibrosis, diabetes, and neurodegenerative diseases. But, function of OGG1 in pulmonary fibrosis was not entirely clear. In this study, we identified a novel function of OGG1 in the cell transformation process in pulmonary fibrosis. We demonstrated that OGG1 and Smad7 co‐localize and interact in A549 cells. Bleomycin‐induced pulmonary fibrosis was established in wild‐type (WT) and Ogg1‐/‐ mice. Upon treatment with transforming growth factor (TGF)‐β1, increased OGG1 expression was observed in WT mice with pulmonary fibrosis as well as in A549 cells, MRC‐5 cells, and primary rat type II alveolar epithelial cells. The increased expression of OGG1 promoted cell migration, while OGG1 depletion decreased migration ability. Expression of the transformation‐associated markers vimentin and alpha‐smooth muscle actin were also affected by OGG1. We also observed that OGG1 promoted TGF‐β1‐induced cell transformation and activated Smad2/3 by interacting with Smad7. The interaction between OGG1 and the TGF‐β/Smad axis modulates the cell transformation process in lung epithelial cells and fibroblasts. Moreover, we demonstrated that Ogg1 deficiency relieved pulmonary fibrosis in bleomycin‐treated mice. Ogg1 knockout decreased the bleomycin‐induced expression of Smad7 and phosphorylation of Smad2/3 in mice. These findings suggest that OGG1 has multiple biological functions in the pathogenesis of pulmonary fibrosis. [ABSTRACT FROM AUTHOR]

Recently, region-based 3D video coding has been proposed. However, existing view synthesis distortion estimation (VSDE) methods are performed at the frame level. To guide the rate-distortion optimization process of region-based 3D video coding schemes, this paper proposes the first pixel-level VSDE (PL-VSDE) method. We first give the definition of the pixel-level view synthesis distortion. To estimate it, a backward prediction method is then developed, which starts from the pixels of interest (POIs) in the virtual view and finds their corresponding pixels in the reference view via a coarse-to-fine approach, denoted as coarse-to-fine backward prediction (CFBP) method. Additionally, the CFBP fully considers the details of 3D warping, the rounding operation and the warping competition in view synthesis, leading to improve accuracy of the prediction. Besides, a table-lookup method and a warping property are introduced to speed up the CFBP. After integrating the CFBP into the PL-VSDE, we can estimate the view synthesis distortion at the pixel level. Our method is carried out pixel-by-pixel independently, which is friendly for parallel processing. The experimental results demonstrate that our proposed method has significant advantages in both accuracy and efficiency compared with the state-of-the-art frame-level VSDE methods. [ABSTRACT FROM AUTHOR]

LiDAR sensors can provide dependable 3D spatial information at a low frequency (around 10 Hz) and have been widely applied in the field of autonomous driving and unmanned aerial vehicle (UAV). However, the camera with a higher frequency (around 20 Hz) has to be decreased so as to match with LiDAR in a multi-sensor system. In this paper, we propose a novel Pseudo-LiDAR interpolation network (PLIN) to increase the frequency of LiDAR sensor data. PLIN can generate temporally and spatially high-quality point cloud sequences to match the high frequency of cameras. To achieve this goal, we design a coarse interpolation stage guided by consecutive sparse depth maps and motion relationship. We also propose a refined interpolation stage guided by the realistic scene. Using this coarse-to-fine cascade structure, our method can progressively perceive multi-modal information and generate accurate intermediate point clouds. To the best of our knowledge, this is the first deep framework for Pseudo-LiDAR point cloud interpolation, which shows appealing applications in navigation systems equipped with LiDAR and cameras. Experimental results demonstrate that PLIN achieves promising performance on the KITTI dataset, significantly outperforming the traditional interpolation method and the state-of-the-art video interpolation technique. [ABSTRACT FROM AUTHOR]

Introduction: Hypertension is associated with increased risk of stroke and bleeding in patients with atrial fibrillation (AF). In the present study, we aimed to investigate the influence of hypertension status in patients with AF receiving treatment with non-vitamin K antagonist oral anticoagulants (NOACs). Methods: PubMed, Embase, and Cochrane Library were searched from the inception of each database to November 2017. Randomized controlled trials (RCTs) that evaluated NOACs versus warfarin in patients with AF and hypertension were identified. A meta-analysis was performed using random- or fixed-effects models according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Results: Five trials (72,967 patients, including 51,378 patients with hypertension) were enrolled. NOACs significantly reduced the risk of stroke and systemic embolism (hazard ratio [HR] 0.87, 95% confidence interval [CI] 0.81–0.94, n = 51,378 patients), hemorrhagic stroke (HR 0.55, 95% CI 0.41–0.74, n = 28,818 patients), death from any cause (HR 0.91, 95% CI 0.85–0.97, n = 43,101 patients), major bleeding (HR 0.78, 95% CI 0.74–0.83, n = 51,378 patients) and intracranial bleeding (HR 0.50, 95% CI 0.38–0.67, n = 27,185 patients). The benefits of NOACs in comparison with warfarin were consistent in AF patients with or without hypertension (Pinteraction for all outcomes > 0.05). Conclusions: Our findings suggest that NOACs can be recommended for the prevention of stroke or systemic embolism in patients with AF and hypertension. [ABSTRACT FROM AUTHOR]

During 3-D video communication, transmission errors, such as packet loss, could happen to the texture and depth sequences. View synthesis distortion will be generated when these sequences are used to synthesize virtual views according to the depth-image-based rendering method. A depth-value-based graphical model (DVGM) has been employed to achieve the accurate packet-loss-caused view synthesis distortion estimation (VSDE). However, the DVGM models the complicated view synthesis processes at depth-value level, which costs too much computation and is difficult to be applied in practice. In this paper, a depth-bin-based graphical model (DBGM) is developed, in which the complicated view synthesis processes are modeled at depth-bin level so that it can be used for the fast VSDE with 1-D parallel camera configuration. To this end, several depth values are fused into one depth bin, and a depth-bin-oriented rule is developed to handle the warping competition process. Then, the properties of the depth bin are analyzed and utilized to form the DBGM. Finally, a conversion algorithm is developed to convert the per-pixel input depth value probability distribution into the depth-bin format. Experimental results verify that our proposed method is 8– $32\times $ faster and requires 17%–60% less memory than the DVGM, with exactly the same accuracy. [ABSTRACT FROM AUTHOR]

Multiview applications endow final users with the possibility to freely navigate within 3D scenes with minimum-delay. A real feeling of scene navigation is enabled by transmitting multiple high-quality camera views, which can be used to synthesize additional virtual views to offer a smooth navigation. However, when network resources are limited, not all camera views can be sent at high quality. It is therefore important, yet challenging, to find the right tradeoff between coding artifacts (reducing the quality of camera views) and virtual synthesis artifacts (reducing the number of camera views sent to users). To this aim, we propose an optimal transmission strategy for interactive multiview HTTP adaptive streaming. We propose a problem formulation to select the optimal set of camera views that the client requests for downloading, such that the navigation quality experienced by the user is optimized while the bandwidth constraints are satisfied. We show that our optimization problem is NP-hard, and we therefore develop an optimal solution based on the dynamic programming algorithm with polynomial time complexity. To further simplify the deployment, we present a suboptimal greedy algorithm with effective performance and lower complexity. The proposed controller is evaluated in theoretical and realistic settings characterized by realistic network statistics estimation, buffer management, and server-side representation optimization. Simulation results show significant improvement in terms of navigation quality compared with alternative baseline multiview adaptation logic solutions. [ABSTRACT FROM AUTHOR]

Interactive multiview video streaming (IMVS) applications permit to freely navigate within 3D scenes. This endows end-users the immersive and virtual reality experiences. Any additional viewpoint of interest can be virtually synthesized by received texture views with associated depth maps. In this paper, we consider the setup where both the encoded texture and depth bitstreams experience packet losses during network streaming. We investigate two packetization strategies and develop a novel strategy to improve error-resilience capabilities for multiview video plus depth transmission, where texture data and the corresponding depth go into the same packet. We examine the resulting distortion of different packetization schemes and analyze the real difference between them in diverse scenarios. In particular, we take into account: (i) various packet loss patterns, where the effect of a burst loss is identified as an important characteristic; (ii) two packet loss models that characterize different probabilities for each scenario. Experimental results demonstrate that, compared with the competing approach, our proposed packetization scheme increases the visual quality by up to 2.32 dB and 1.55 dB for texture views and synthesized views, respectively. [ABSTRACT FROM AUTHOR]

The resolution of the depth map captured by a time-of-flight camera is rather limited; hence, it cannot be directly used to build a high-quality 3D model, together with an associated high-resolution texture image. In this paper, an improved depth up-sampling algorithm that considers two important characteristics of the depth map is proposed. First, the invariance of the fractal dimension between the low-resolution and high-resolution depth maps is maintained. Second, the boundaries of the depth map and its corresponding texture image are made to be consistent to provide a good synthesized virtual view. The experimental results show that the proposed method achieves superior performance for the up-sampled depth map and synthesized virtual view compared with classical approaches. [ABSTRACT FROM AUTHOR]

[Display omitted] • YOLOv5 network structure was adapted for the small target detection scenario. • A universal deep learning block integrating ensemble and attention mechanisms suitable for multitasking. • Circular smooth label is introduced to detect the rotated object efficiently. Object detection technology for images generated by optical sensors is of great significance in areas such as national defense security, disaster prediction, and smart city construction. Aiming at the problems of small target size, arbitrary target direction, and complex background in aerial image object detection using optical sensors. By using the images captured by a wild range of optical sensors such as CCD（Charge Coupled Device） and SAR(Synthetic Aperture Radar), we propose a more efficient rotating frame object detection algorithm that introduces the MCAB (Multi-branch Convolutional Attention Block) that can process these same types of images taken by the aerial platform using light sensors. First, we construct the module by adding identity, residual branch, and CBAM (Convolutional Block Attention Module) structure to the traditional convolution layer and the activation function to substitute the simple conv module in the network. Secondly, we introduce the Transformer layer at the end of the backbone network to enhance the global perception of the model with low overhead, realize the relationship is modeling between the target and the scene content, and reduce the amount of calculation. We improve the structure of the neck layer by BiFPN (Bidirectional Feature Pyramid Network) to speed up network operation, and CBAM is added after C3 to highlight important characteristic information. Our improved YOLOv5 algorithm is tested on the self-made rotating small target aerial data set. Compared with the original algorithm, our model's mean Average Precision (mAP) is improved by 1.8 percentage points, and the accuracy and recall are also enhanced by 0.46 and 0.34 percentage points. [ABSTRACT FROM AUTHOR]

• Change of the components in CS after HO pretreatment is quantitatively measured. • HO pretreatment improves the quality of biochar pellets and wood vinegar. • The enrichment of hemicellulose and cellulose conrtibutes to the upgrading effect. • Pretreatment at 180 °C is best for CS to produce biochar pellets and wood vinegar. In the present study, biochar pellets and wood vinegar are prepared from the cotton stalk (CS) through the densification and pyrolysis, and hydrothermal oxidative (HO) pretreatment (160–260 °C) is conducted to CS to upgrade the quality of biochar pellet and wood vinegar (WV). The results show that HO pretreatment can decompose the hemicellulose, amorphous cellulose, crystalline cellulose, and lignin at 160–230 °C, 160–200 °C, 200–260 °C, and < 200 °C, respectively. Therefore, the hemicellulose and cellulose (especially the crystalline cellulose) in CS are enriched at 160–180 °C and 160–200 °C respectively. Due to the enrichment of hemicellulose and cellulose, the organic compounds in WV increase dramatically after HO pretreatment, resulting in a significant increase in the density and acidity of WV. Furthermore, due to the enrichment of hemicellulose and cellulose, the biochar pellets exhibit more densified internal structures after HO pretreatment, contributing to a higher relaxation density, energy density, and compressive strength. Moreover, the biochar pellets exhibit better combustion performance and lower ash content after HO pretreatment at 160–180 °C. The CS sample pretreated at 180 °C is the best material for the production of biochar pellets and WV. [ABSTRACT FROM AUTHOR]

• High-throughput screening of active components of Xiaoke pills in vivo. • Rapid research strategy of TCM's in vivo ADME process. • First report on complete and efficient ADME analysis strategy of TCM for zebrafish. Traditional Chinese medicine (TCM) has been used in clinical settings for over 2000 years in China. The study of the absorption, distribution, metabolism, and excretion (ADME) of TCM in vivo could be beneficial for the discovery of the active components in TCM. However, the conventional strategies used for ADME research are based on rodent models and have the characteristics of lengthy experimental periods, complex processes, and extensive data processing, which make it difficult to perform rapid analyses and high-throughput ADME screening of the medicinal components of TCM. In this study, an integrated high-throughput research strategy for the in vivo ADME analysis of TCM was established based on a zebrafish model. Accordingly, a combination of ultra-performance liquid chromatography-high-resolution mass spectrometry (UPLC-HRMS), desorption electrospray ionization-mass spectrometry (DESI-MS) imaging, and in-house non-targeted precise-and-thorough background-subtraction (PATBS) data post-processing techniques were successfully applied for the analysis of the metabolism of zebrafish exposed to Xiaoke pills. A total of 49 compounds related to Xiaoke pills (including 13 prototypical components and 36 metabolites) were detected in zebrafish. In total, 32 of them, including puerarin, daidzein, deoxyschizandrin, formononetin, and glibenclamide, which have been identified to have hypoglycemic activity in our previous studies and are phase I and phase II metabolites resulting from the hydroxylation, demethylation, glucuronidation, sulfation, and glycosylation of the prototypical components in vivo , were found in rats treated with Xiaoke pills. Furthermore, the overall distribution of the known compounds in zebrafish exposed to Xiaoke pills was explored using DESI-MS. In summary, this study provides a practical approach for the high-throughput screening of the active components of TCM using a zebrafish model. [ABSTRACT FROM AUTHOR]

Hydrothermal pretreatment (HT) is an effective method to upgrade biochar pellets, while little attention is paid to wood vinegar (WV), the byproduct of biochar pellets production. Therefore, Chinese fir sawdust (FS) and cotton stalk (CS) were hydrothermally pretreated first at 180–280 °C and then pyrolyzed at 350 °C in the study. The effect of hydrothermal pretreatment on the properties of biomass was evaluated by the Van Soest method and thermogravimetric method. The yield, pH, density, and composition of WV were analyzed. The results showed that hemicellulose and cellulose were completely decomposed at HT temperatures of 230 °C and 280 °C respectively. The removal of hemicellulose and the enrichment of cellulose and lignin in the samples pretreated before 230 °C result in a higher density and lower pH of WV, while the yield only increases before 200 °C. Compared with CS samples, the higher content and reactivity of cellulose, especially amorphous cellulose, and lignin in FS samples results in the higher yield of the organic compounds in FSWV. With HT temperature increasing to 230 °C, the content and yield of acids in FSWV/CSWV shows a declining trend, and the content and yield of phenols, ketones, and aldehydes increase. HT pretreatment below 230 °C can upgrade WV. • Evolution of the components in FS and CS during HT pretreatment is given. • Relationship between the components of FS/CS samples and WV is gained. • Less hemicellulose and more amorphous cellulose and lignin improve WV quality. • FS with higher reactivity is more suitable for WV preparation than CS. • WV can be upgraded from the biomass HT pretreated below 230 °C. [ABSTRACT FROM AUTHOR]

The depth-image-based-rendering is a key technique to realize free viewpoint television. However, one critical problem in these systems is filling the disocclusion due to the 3-D warping process. This paper exploits the temporal correlation of texture and depth information to generate a background reference image. This is then used to fill the holes associated with the dynamic parts of the scene, whereas for static parts the traditional inpainting method is used. To generate the background reference image, the Gaussian mixture model is employed on the texture information, whereas, depth maps information are used to detect moving objects so as to enhance the background reference image. The proposed holes filling approach is particularly useful for the single-view-plus-depth format, where, contrary to the multi-view-plus-depth format, only information of one view could be used for this task. The experimental results show that objective and subjective gains can be achieved, and the gain ranges from 1 to 3 dB over the inpainting method. [ABSTRACT FROM PUBLISHER]