Search

Abstract Point cloud data is a large collection of high dimensional 3D points with 3D coordinates and attributes, which has been one of the mainstream representations for emerging 3D applications, such as virtual reality, autonomous vehicles, and robotics. Due to the large‐scale unstructured high‐dimensional nature of point clouds, point cloud processing, transmitting and analysing has been challenging issues in multimedia signal processing and communication. Deep learning is a powerful tool to learn statistical knowledge from massive data. Advances in artificial intelligence, especially deep learning models are offering new opportunities for point cloud processing, compression and analysis. This special issue aims at promoting cutting‐edge research on deep learning‐based point cloud processing, including object detection, segmentation, registration, compression, and visual quality assessment.

Point clouds are sets of points used to visualize three-dimensional (3D) objects. Point clouds can be static or dynamic. Each point is characterized by its 3D geometry coordinates and attributes such as color. High-quality visualizations often require millions of points, resulting in large storage and transmission costs, especially for dynamic point clouds. To address this problem, the moving picture experts group has recently developed a compression standard for dynamic point clouds called video-based point cloud compression (V-PCC). The standard generates two-dimensional videos from the geometry and color information of the point cloud sequence. Each video is then compressed with a video coder, which converts each frame into frequency coefficients and quantizes them using a quantization parameter (QP). Traditionally, the QPs are severely constrained. For example, in the low-delay configuration of the V-PCC reference software, the quantization parameter values of all the frames in a group of pictures are set to be equal. We show that the rate-distortion performance can be improved by relaxing this constraint and treating the QP selection problem as a multi-variable constrained combinatorial optimization problem, where the variables are the QPs. To solve the optimization problem, we propose a variant of the differential evolution (DE) algorithm. Differential evolution is an evolutionary algorithm that has been successfully applied to various optimization problems. In DE, an initial population of randomly generated candidate solutions is iteratively improved. At each iteration, mutants are generated from the population. Crossover between a mutant and a parent produces offspring. If the performance of the offspring is better than that of the parent, the offspring replaces the parent. While DE was initially introduced for continuous unconstrained optimization problems, we adapt it for our constrained combinatorial optimization problem. Also, unlike standard DE, we apply individual mutation to each variable. Furthermore, we use a variable crossover rate to balance exploration and exploitation. Experimental results for the low-delay configuration of the V-PCC reference software show that our method can reduce the average bitrate by up to 43% compared to a method that uses the same QP values for all frames and selects them according to an interior point method.

This special section contains extended versions of papers presented at the IEEE International Conference on Multimedia & Expo (ICME) in London, July 2020. The invitations to contribute to the special section were based on the quality of the conference paper and its relevance to the aims and scope of the journal. Each submitted manuscript was reviewed by at least three anonymous reviewers. After a rigorous review process, three papers were accepted. Two of them present new developments in immersive media, while the third deals with the problem of efficiently encoding a deep neural network.