Search

This paper aims to investigate the ambient contrast ratio of transparent displays in AR/MR applications. Transparent OLED displays in indoor environments have been widely applied. However, the high ambient brightness in outdoor environments limits the application field due to poor ambient contrast of OLED. Thanks to the high brightness of the transparent Micro‐LED display, AR/MR applications can be implemented from indoors to outdoors. In this paper, we try to find the appropriate ambient contrast ratio (ACR) range, that is, the viewer sees the appropriate background image and display information simultaneously under different environmental conditions. In addition to using direct transparent display, a switchable black cell is used to extend the adaptive ACR system. Therefore, the proposed ACR control algorithm is demonstrated in a combination of transparent Micro‐LED displays with a switchable black cell to discuss extending the application field from low to high ambient brightness. [ABSTRACT FROM AUTHOR]

Eight new 11-methoxymethylgermacranolides (1–8) were isolated from the ethanol extract of the whole plant of Carpesium divaricatum. The planar structures and relative configurations of the new compounds were determined by detailed spectroscopic analysis. The absolute configuration of 1 was established by electronic circular dichroism (ECD) spectrum and X-ray crystallographic analysis, and the stereochemistry of the new compounds 2–8 were determined by similar ECD data with 1. The absolute configurations of 5 and 7 were further confirmed by using quantum chemical electronic circular dichroism (ECD) calculations. Compound 4 exhibited weak cytotoxicity against MCF-7 cells. Compound 8 could potently decrease PGE2 productions in LPS-induced RAW 264.7 cells. [ABSTRACT FROM AUTHOR]

Homography estimation is an important task in computer vision applications, such as image stitching, video stabilization, and camera calibration. Traditional homography estimation methods heavily depend on the quantity and distribution of feature correspondences, leading to poor robustness in low-texture scenes. The learning solutions, on the contrary, try to learn robust deep features but demonstrate unsatisfying performance in the scenes with low overlap rates. In this paper, we address these two problems simultaneously by designing a contextual correlation layer (CCL). The CCL can efficiently capture the long-range correlation within feature maps and can be flexibly used in a learning framework. In addition, considering that a single homography can not represent the complex spatial transformation in depth-varying images with parallax, we propose to predict multi-grid homography from global to local. Moreover, we equip our network with a depth perception capability, by introducing a novel depth-aware shape-preserved loss. Extensive experiments demonstrate the superiority of our method over state-of-the-art solutions in the synthetic benchmark dataset and real-world dataset. The codes and models will be available at https://github.com/nie-lang/Multi-Grid-Deep-Homography. [ABSTRACT FROM AUTHOR]

Pseudo-LiDAR point cloud interpolation is a novel and challenging task in autonomous driving, which aims to address the frequency mismatching problem between a camera and a LiDAR. Previous works represent the 3D spatial motion relationship with a coarse 2D optical flow, and the quality of interpolated point clouds only depends on the supervision of depth maps. As a result, the generated point clouds suffer from inferior global distributions and local appearances. To solve the above problems, we propose a Pseudo-LiDAR point cloud interpolation network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow from point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship. For a more comprehensive perception of the distribution of a point cloud, we design a novel reconstruction loss function with the chamfer distance to supervise the generation of Pseudo-LiDAR point clouds in 3D space. In addition, we introduce a multi-modal deep aggregation module to facilitate the efficient fusion of texture and depth features. As the benefits of the improved motion representation, training loss function, and model structure, our approach gains significant improvements on the Pseudo-LiDAR point cloud interpolation task. The experimental results evaluated on KITTI dataset demonstrate the state-of-the-art quantitative and qualitative performance of the proposed network. [ABSTRACT FROM AUTHOR]

Fisheye cameras can capture a large field-of-view (Fov) scene but it introduces severe radial distortion in images. Thus, distortion rectification is a crucial step for subsequent computer vision tasks using fisheye cameras. A prevalent type of method predicts the displacement field between the input and output to rectify the distorted images. However, it is challenging to estimate the accurate flow in Cartesian coordinates (both $x$ and $y$ directions), in which the sampling strategy of the convolution kernel ignores the radial symmetry of distortion. In general, the pixel’s distortion at the same radius from the center is the same, while the radius corresponds to one parameter in polar coordinates. Motivated by this fact, we exploit the radial symmetry of distortion to predict a more straightforward one-dimensional flow, transforming the distorted image into the polar coordinates domain instead of predicting two-dimensional flow in $x$ and $y$ directions. Specifically, we propose a Polar coordinates Distortion Rectification Network (PCDRN), whose sampling strategy corresponds to the radial distortion characteristic so that a more accurate flow can be predicted. To eliminate the blurs and ring artifacts induced by the coordinates transformation, a Polar-To-Cartesian Appearance Enhancement Network is designed to enhance the local appearance of rectified images. Experimental results on the synthesized dataset and real-world dataset demonstrate the superiority of our approach in both quantitative and qualitative evaluations. [ABSTRACT FROM AUTHOR]

Traditional feature-based image stitching technologies rely heavily on feature detection quality, often failing to stitch images with few features or low resolution. The learning-based image stitching solutions are rarely studied due to the lack of labeled data, making the supervised methods unreliable. To address the above limitations, we propose an unsupervised deep image stitching framework consisting of two stages: unsupervised coarse image alignment and unsupervised image reconstruction. In the first stage, we design an ablation-based loss to constrain an unsupervised homography network, which is more suitable for large-baseline scenes. Moreover, a transformer layer is introduced to warp the input images in the stitching-domain space. In the second stage, motivated by the insight that the misalignments in pixel-level can be eliminated to a certain extent in feature-level, we design an unsupervised image reconstruction network to eliminate the artifacts from features to pixels. Specifically, the reconstruction network can be implemented by a low-resolution deformation branch and a high-resolution refined branch, learning the deformation rules of image stitching and enhancing the resolution simultaneously. To establish an evaluation benchmark and train the learning framework, a comprehensive real-world image dataset for unsupervised deep image stitching is presented and released. Extensive experiments well demonstrate the superiority of our method over other state-of-the-art solutions. Even compared with the supervised solutions, our image stitching quality is still preferred by users. [ABSTRACT FROM AUTHOR]

Radial distortion has widely existed in the images captured by popular wide-angle cameras and fisheye cameras. Despite the long history of distortion rectification, accurately estimating the distortion parameters from a single distorted image is still challenging. The main reason is that these parameters are implicit to image features, influencing the networks to learn the distortion information fully. In this work, we propose a novel distortion rectification approach that can obtain more accurate parameters with higher efficiency. Our key insight is that distortion rectification can be cast as a problem of learning an ordinal distortion from a single distorted image. To solve this problem, we design a local-global associated estimation network that learns the ordinal distortion to approximate the realistic distortion distribution. In contrast to the implicit distortion parameters, the proposed ordinal distortion has a more explicit relationship with image features, and significantly boosts the distortion perception of neural networks. Considering the redundancy of distortion information, our approach only uses a patch of the distorted image for the ordinal distortion estimation, showing promising applications in efficient distortion rectification. In the distortion rectification field, we are the first to unify the heterogeneous distortion parameters into a learning-friendly intermediate representation through ordinal distortion, bridging the gap between image feature and distortion rectification. The experimental results demonstrate that our approach outperforms the state-of-the-art methods by a significant margin, with approximately 23% improvement on the quantitative evaluation while displaying the best performance on visual appearance. [ABSTRACT FROM AUTHOR]

Ultra-high definition (UHD) 360 videos encoded in fine quality are typically too large to stream in its entirety over bandwidth (BW)-constrained networks. One popular approach is to interactively extract and send a spatial sub-region corresponding to a viewer’s current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction foretelling a viewer’s future FoVs is essential. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem: three sources of relevant information—collected viewers’ head movement traces, a 360 image saliency map, and a biological human head model—are distilled into a view transition Markov model. Specifically, we formulate a constrained maximum a posteriori (MAP) problem with likelihood and prior terms defined using the three information sources. We solve the MAP problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe (FW) optimization strategy. In each FW iteration, a linear program (LP) is solved, whose runtime is reduced thanks to warm start initialization. Having estimated a Markov model from data, we employ it to optimize a tile-based 360 video streaming system. Extensive experiments show that our head movement prediction scheme noticeably outperformed existing proposals, and our optimized tile-based streaming scheme outperformed competitors in rate-distortion performance. [ABSTRACT FROM AUTHOR]

The microRNAs (miRNAs) in circulating small extracellular vesicles (sEVs) have been suggested as potential biomarkers in cancer diagnosis. This study was designed to evaluate the circulating sEV‐derived miRNAs as biomarkers for the diagnosis of nasopharyngeal carcinoma (NPC). We compared the miRNA profiles in plasma‐derived sEVs between 16 patients with NPC and 5 healthy controls (HCs). A distinct set of miRNAs that were differentially expressed between patients with NPC and HCs was determined by means of integrative bioinformatics approaches. Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment pathway analysis revealed that the target genes of the differentially expressed miRNAs (DEMs) were mainly involved in cancer‐associated signaling pathways. Seven representative DEMs were selected and further validated in an additional 60 patients with NPC and 40 HCs using quantitative reverse‐transcription PCR analysis (qRT‐PCR). Receiver operating characteristic (ROC) curve analysis was used to assess the accuracy of the sEV‐miRNA‐based model for diagnosis. The 3 miRNA‐based model, comprising miR‐134‐5p, miR‐205‐5p, and miR‐409‐3p, showed good discriminating power with an area under the curve (AUC) value of 0.88 in the training set and 0.91 in the validation set. Furthermore, the diagnostic model had an excellent classification ability to distinguish patients with NPC at different clinical stages or Epstein‐Barr virus infection status from HCs. In conclusion, our findings indicated that sEV‐derived miRNA levels were altered in the plasma of patients with NPC in comparison with those in HCs. The model based on the 3 sEV‐derived miRNAs could potentially act as an alternative or complementary approach for diagnosing NPC. This study was designed to evaluate circulating sEV‐derived miRNAs as biomarkers for the diagnosis of nasopharyngeal carcinoma (NPC). Our findings indicated that sEV‐derived miRNA levels were altered in the plasma of patients with NPC compared with those in HCs. The model based on the 3 sEV‐derived miRNAs could potentially act as an alternative or complementary approach for diagnosing NPC. [ABSTRACT FROM AUTHOR]

Abstract Intra macroblock refreshment is an effective approach for error-resilient video coding. In this paper, in addition to intra coding, we propose to add two macroblock coding modes to enhance the transmission robustness of the coded bitstream, which are inter coding with redundant macroblock and intra coding with redundant macroblock. The selection of coding modes and the parameters for coding the redundant version of the macroblock are determined by the rate-distortion optimization. It is worth mentioning that the end-to-end distortion is employed in the optimization procedure, which considers the channel conditions. Extensive simulation results show that the proposed approach outperforms other error-resilient approaches significantly; for some video sequences, the average PSNR can be up to 4 dB higher than that of the Optimal Intra Refreshment approach.

Abstract This paper proposes a scheme for error-resilient transmission of videos which jointly uses intra macroblock refreshment and redundant motion vector. The selection of using intra refreshment or redundant motion vector is determined by the rate-distortion optimization procedure. The end-to-end distortion is used for the rate-distortion optimization, which can be easily calculated with the recursive optimal per-pixel estimate (ROPE) method. Simulation results show that the proposed method outperforms both the intra refreshment approach and redundant motion vector approach significantly, when the two approaches are deployed separately. Specifically, for the Foreman sequence, the average PSNR of the proposed approach can be 1.12 dB higher than that of the intra refreshment approach and 5 dB higher than that of the redundant motion vector approach.

Distortion rectification is a fundamental task in the field of computer vision and image processing. Nevertheless, previous methods have regarded distortion rectification as a static problem that learns a mapping function and corrects the distorted image to a unique state. However, this state is generally not the optimal solution, as it would result in an under-rectified or over-rectified structure. In this study, we revisit the classical distortion rectification task with a new perspective and redesign the algorithm, inspired by video processing techniques. Specifically, we regard distortion rectification as a dynamic problem that can be extended to a sequence of different distortion states: the input distorted image (t), under-rectified image (t+1), ideal-rectified image (t+2), and over-rectified image (t+3). We first estimate the residual distortion map (RDM) between the input distorted image and the coarse-rectified (t+1 or t+3) image. Here, RDM indicates the motion difference between two distorted images. Subsequently, the RDM is used to guide the refinement rectification process, aiming to convert the coarse-rectified state into the ideal-rectified state. In addition, the flexible implementation of the proposed refinement process with RDM to improve the rectification results of any method is appealing. The experimental results demonstrate that our method outperforms the state-of-the-art schemes by a significant margin, revealing approximately 40% improvement through quantitative evaluation. [ABSTRACT FROM AUTHOR]

360° video is very popular due to its 360° views of a scene. Although 360° videos are also compressed by a hybrid coding framework like 2D video, its high resolution and serious shape deformation affect coding efficiency. In equirectangular projection (ERP) format of 360° videos, if an object moves from equator regions to pole regions or vice versa, large deformation will be introduced and motion estimation cannot find the best‐matched part. To solve the above problem, the authors propose to generate a better reference frame for the current to be encoded frame. First, they project the frame prior to the current one from ERP to the sphere and rotate it at an appropriate angle depending on motion vectors. Subsequently, they insert this generated frame to the rear of the reference queue and let the encoder work as usual. The advantage is that the inserted frame has a more similar shape deformation as the current frame, which greatly helps motion estimation and makes full use of 360° video characters. Their method is simple and friendly compatible with the existing compression standard. Experiments prove that their method achieves 1.57% Bjøntegaard Delta (BD)‐gain compared with standard high efficiency video coding. [ABSTRACT FROM AUTHOR]

Recently, learning-based distortion rectification schemes have shown high efficiency. However, most of these methods only focus on a specific camera model with fixed parameters, thus failing to be extended to other models. To avoid such a disadvantage, we propose a model-free distortion rectification framework for the single-shot case, bridged by the distortion distribution map (DDM). Our framework is based on an observation that the pixel-wise distortion information is explicitly regular in a distorted image, despite different models having different types and numbers of distortion parameters. Motivated by this observation, instead of estimating the heterogeneous distortion parameters, we construct a proposed distortion distribution map that intuitively indicates the global distortion features of a distorted image. In addition, we develop a dual-stream feature learning module, benefitting from both the advantages of traditional methods that leverage the local handcrafted feature and learning-based methods that focus on the global semantic feature perception. Due to the sparsity of handcrafted features, we discrete the features into a 2D point map and learn the structure inspired by PointNet. Finally, a multimodal attention fusion module is designed to attentively fuse the local structural and global semantic features, providing the hybrid features for the more reasonable scene recovery. The experimental results demonstrate the excellent generalization ability and more significant performance of our method in both quantitative and qualitative evaluations, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]