Search

For point cloud registration, the purpose of this article is to propose a novel centralized random sample consensus (RANSAC) (C-RANSAC) registration with fast convergence and high accuracy. In our algorithm, the novel contributions are, first, the proposal of a scale histogram-based outlier removal to delete outliers from the initial line vector set $\mathbf {L}$ for constructing a reduced line vector set $\mathbf {L}_{\text{red}}$; second, the handshake cooperation between the host RANSAC (H-RANSAC) only working on L and the local RANSAC (LCL-RANSAC) only working on $\mathbf {L}_{\text{red}}$; third, in each handshake process, after receiving the global registration solution and the global iteration number $x_{H}$ from H-RANSAC, LCL-RANSAC uses the received global solution as the initial solution of the modified TEASER++ (M-TEASER++) method to calculate its first local registration solution. If the first local registration solution satisfies the global iteration number inheritance condition, LCL-RANSAC directly sends the accumulated iteration number, $x_{H} + 1$, and the first local solution back to H-RANSAC; otherwise, LCL-RANSAC iteratively refines its local solution using the M-TEASER++ method, and then sends the resultant local solution and the required local iteration number $x_{\text{LCL}}$ to H-RANSAC for updating the global solution, the global iteration number to $x_{H} := x_{H} + x_{\text{LCL}}$, and the global confidence level. Due to $|L_{\text{red}}| \ll |L|$ and employing the global iteration number inheritance condition test into our algorithm, we have conducted extensive experiments on testing point cloud pairs to show the registration accuracy and execution time merits of our algorithm relative to the state-of-the-art methods.

The Bayer and RGBW color filter array (CFA) raw images, denoted by $I^{Bayer}$ and $I^{RGB}$ , respectively, have been widely used in consumer markets. In the demosaicking-first compression scheme, chroma subsampling is necessary prior to compressing $I^{RGB}$ and $I^{RGBW}$ . Several linear interpolation-based chroma subsampling methods have been developed for $I^{Bayer}$ , but no nonlinear interpolation-based chroma subsampling methods have targeted the above two CFA image types simultaneously. In this paper, we first propose a nonlinear interpolation-based, namely the cubic convolution interpolation-based (CCI-based), $2\times 2$ block-distortion function for each $2\times 2$ CFA block $B^{CbCr}$ . Next, using the Cauchy-Schwarz inequality, we prove that the proposed block-distortion function is a convex function in the real domain, which further serves as the base of the initial subsampled chroma solution. Then, a CCI-based iterative method is proposed to improve the initial subsampled chroma solution. The results of comprehensive experimental tests using Bayer and RGBW CFA images created from the three RGB full-color datasets, namely IMAX, SCI (screen content images), and CI (classical images), demonstrate that on the Versatile Video Coding (VVC) platform VTM-11.0, the proposed method achieves substantial quality enhancement and quality-bitrate tradeoff merits of the reconstructed Bayer and RGBW CFA images compared with existing chroma subsampling methods.

Traditionally, prior to compressing an RGB full-color image, for each converted $2\times 2$ CbCr block $B^{CbCr}$ , chroma subsampling only downsamples $B^{CbCr}$ , but without changing the luma block $B^{Y}$ at all. In the current research, a special linear interpolation-based, namely the COPY-based, chroma subsampling-first luma modification (CSFLM) study has attempted to change the luma block for enhancing the quality of the reconstructed RGB full-color image. In this paper, a fast and effective nonlinear interpolation, namely the bicubic convolution interpolation (BCI), based iterative luma modification method for CSFLM is proposed. In our iterative method, a BCI-based distortion function and its convex property proof are first provided. Next, based on the proposed convex distortion function, a pseudoinverse technique is applied to obtain the initial luma modification solution, and then an iterative method is proposed to improve the initial luma modification solution. Based on five testing image datasets, namely the IMAX, Kodak, SCI (screen content images), CI (classical images), and Video datasets, the thorough experimental results have demonstrated that on the newly released Versatile Video Coding (VVC) platform VTM-12.0, our iterative luma modification method achieves substantial quality, execution-time, and quality-bitrate tradeoff improvements when compared with the existing state-of-the-art methods.

The Bayer color filter array (CFA) pattern has been widely used in modern digital color cameras, and the captured image is called the Bayer CFA image IBayer. Chroma downsampling is a necessary and important step in the coding system. In this paper, a novel and optimal luma modification-based chroma downsampling (LMCD) method is proposed for IBayer. For each converted 2 x 2 YUV block, the downsampled (U, V)-pair and the modified luma values by the proposed LMCD method can be optimally determined, and the optimal variant of our LMCD method associated with the two modified luma values has the best quality among all sixteen variants of LMCD. Based on the Kodak and IMAX datasets, on the High Efficiency Video Coding (HEVC) platform HM-16.18, the Versatile Video Coding (VVC) platform VTM-8.0, and the Joint Photographic Experts Group (JPEG) platform, the comprehensive experimental data showed the substantial quality and quality-bitrate tradeoff merits of the reconstructed images by our optimal LMCD method relative to the traditional and state-of-the-art chroma downsampling methods.

Differing from the traditional chroma subsampling on the YUV image converted from a RGB full-color image, in this paper, we propose a novel and effective chroma subsampling and luma modification (CSLM) method. For each 2 × 2 YUV block, first, a newly reconstructed 2 × 2 RGB full-color blockdistortion model is proposed, and then we propose a multiple linear regression approach to tackle our CSLM method such that the reconstructed 2 × 2 RGB full-color block-distortion can be minimized, achieving significant quality improvement of the reconstructed RGB full-color image. Based on the Kodak and IMAX datasets, the comprehensive experimental results demonstrated that on the versatile video coding (VVC) platform VTM-8.0, our method achieves substantial quality and quality-bitrate tradeoff improvement of the reconstructed RGB full-color images relative to six traditional methods and the three state-of-the-art methods.

In this paper, we propose an effective source-aware domain enhancement and adaptation (SDEA) approach to increase the accuracy of the existing convolutional neural network-based (CNN-based) object segmentation methods. We first scoop out the source elements, such as the falling-leaves, manhole covers, cirrus clouds, and advertisements, which often cause invalid object segmentation and make the existing object segmentation methods provide unreliable information to the ADAS (automatic driving assistance systems) applications. Secondly, we create a new GTA5-like (Grand Theft Auto V-like) dataset with the scenarios including these source elements. Furthermore, we perform a domain adaptation on the created GTA5-like dataset to generate a photo-realistic GTA5-like dataset, namely GTA5sSDEA. Without the need to relabel the pixel-annotations for GTA5sSDEA, we combine GTA5sSDEA with the realistic dataset, namely Camvid, to constitute a newly enhanced dataset. After retraining the existing CNN-based object segmentation methods by using our enhanced dataset, it can achieve substantial segmentation accuracy improvement. The comprehensive experimental results have demonstrated the clear accuracy improvement merit by applying our SDEA approach to the state-of-the-art object segmentation methods on FCN (Fully Convolutional Networks), SegNet-basic, AdaptSegNet, and Gated-AdaptSegNet, providing more reliable information to ADAS applications.

Given a source UAV (unmanned aerial vehicle) image Is and a target UAV image It, it is a challenging problem to correct the color of all target pixels so that the subjective and objective quality effects between Is and It can be as consistent as possible. Recently, by referring to all stitching color difference values on the stitching line, a global bilateral joint interpolation-based (GBJI-based) color correction method was proposed. However, because all stitching color difference values may contain aligned and misaligned stitching pixels, the GBJI-based method suffers from a perceptual artifact near the misaligned stitching pixels. To remedy this perceptual artifact, in this paper, we propose an adaptive joint bilateral interpolation-based (AJBI-based) color blending method such that each target pixel only adaptively refers to an adequate interval of stitching color difference values locally. Based on several testing stitched UAV images under different brightness and misalignment situations, comprehensive experimental results demonstrate that in terms of PSNR (peak signal-to-noise ratio), SSIM (structural similarity index), and FSIM (feature similarity index), our method achieves higher objective quality effects and also achieves better perceptual effects, particularly near the misaligned stitching pixels, when compared with the GBJI-based method and the other state-of-the-art methods.

Bayer color filter array (CFA) images are captured by a single-chip image sensor covered with a Bayer CFA pattern which has been widely used in modern digital cameras. In the past two decades, many compression methods have been proposed to compress Bayer CFA images. These compression methods can be roughly divided into the compression-first-based (CF-based) scheme and the demosaicing-first-based (DF-based) scheme. However, in the literature, no review article for the two compression schemes and their compression performance is reported. In this article, the related CF-based and DF-based compression works are reviewed first. Then, the testing Bayer CFA images created from the Kodak, IMAX, screen content images, videos, and classical image datasets are compressed on the Joint Photographic Experts Group-2000 (JPEG-2000) and the newly released Versatile Video Coding (VVC) platform VTM-16.2. In terms of the commonly used objective quality, perceptual quality metrics, the perceptual effect, and the quality–bitrate tradeoff metric, the compression performance comparison of the CF-based compression methods, in particular the reversible color transform-based compression methods and the DF-based compression methods, is reported and discussed.

The success of many computer vision and pattern recognition applications depends on matching local features on two or more images. Because the initial correspondence set—i.e., the set of the initial feature pairs—is often contaminated by mismatches, removing mismatches is a necessary task prior to image matching. In this paper, we first propose a fast geometry histogram-based (GH-based) mismatch removal strategy to construct a reduced correspondence set Creduced,GH from the initial correspondence set Cini. Next, we propose an effective cooperative random sample consensus (COOSAC) method for remote sensing image matching. COOSAC consists of a RANSAC, called RANSACini working on Cini, and a tiny RANSAC, called RANSACtiny,GH working on a randomly selected subset of Creduced,GH. In RANSACtiny,GH, an iterative area constraint-based sampling strategy is proposed to estimate the model solution of Ctiny,GH until the specified confidence level is reached, and then RANSACini utilizes the estimated model solution of Ctiny,GH to calculate the inlier rate of Cini. COOSAC repeats the above cooperation between RANSACtiny,GH and RANSACini until the specified confidence level is reached, reporting the resultant model solution of Cini. For convenience, our image matching method is called the GH-COOSAC method. Based on several testing datasets, thorough experimental results demonstrate that the proposed GH-COOSAC method achieves lower computational cost and higher matching accuracy benefits when compared with the state-of-the-art image matching methods.

As the color filter array (CFA)2.0, the RGBW CFA pattern, in which each CFA pixel contains only one R, G, B, or W color value, provides more luminance information than the Bayer CFA pattern. Demosaicking RGBW CFA images I R G B W is necessary in order to provide high-quality RGB full-color images as the target images for human perception. In this letter, we propose a three-stage demosaicking method for I R G B W . In the first-stage, a cross shape-based color difference approach is proposed in order to interpolate the missing W color pixels in the W color plane of I R G B W . In the second stage, an iterative error compensation-based demosaicking process is proposed to improve the quality of the demosaiced RGB full-color image. In the third stage, taking the input image I R G B W as the ground truth RGBW CFA image, an I R G B W -based refinement process is proposed to refine the quality of the demosaiced image obtained by the second stage. Based on the testing RGBW images that were collected from the Kodak and IMAX datasets, the comprehensive experimental results illustrated that the proposed three-stage demosaicking method achieves substantial quality and perceptual effect improvement relative to the previous method by Hamilton and Compton and the two state-of-the-art methods, Kwan et al.’s pansharpening-based method, and Kwan and Chou’s deep learning-based method.

The purpose of this research is to develop multi-talented humanoid robots, based on technologies featuring high-computing and control abilities, to perform onstage. It has been a worldwide trend in the last decade to apply robot technologies in theatrical performance. The more robot performers resemble human beings, the easier it becomes for the emotions of audiences to be bonded with robotic performances. Although all kinds of robots can be theatrical performers based on programs, humanoid robots are more advantageous for playing a wider range of characters because of their resemblance to human beings. Thus, developing theatrical humanoid robots is becoming very important in the field of the robot theatre. However, theatrical humanoid robots need to possess the same versatile abilities as their human counterparts, instead of merely posing or performing motion demonstrations onstage, otherwise audiences will easily become bored. The four theatrical robots developed for this research have successfully performed in a public performance and participated in five programs. All of them were approved by most audiences.