Search

[Display omitted] • This special section includes papers on computer graphics education. • These papers are the best papers from the Eurographics 2022 Education Papers Program. • Specific topics include ray-tracing, the Vulkan API, and virtual reality. [ABSTRACT FROM AUTHOR]

This research investigated the space compositions of Chernikhov’s 101 Architectural Fantasies via computer-aided simulation to interpret the relationships between architectural components and spatial organization. An algorithmic approach and a perception approach were tested. Traditional analysis emphasized the simulation of corresponding objects by perspective deconstruction methods, which might not be able to show the exactly correct spatial relationship between objects. This research adopted photogrammetry to investigate the non-orthogonal spatial construction of 3D objects in 2D pictures. Research results showed that the algorithmic approach may derive different degrees of angles of parallel or intersected objects, and that observers tend to be misled by the effect of “orthogonal assumption” in terms of their own visual experiences. This finding revealed that Chernikhov had created unreasonable descriptions of space. This result was verified by the existence of false parallel and orthogonal relationships between drawn building parts. Three tests were conducted. Observers used a reverse verification process to analyze three-dimensional objects re-built in simulation. The verification mirrored a two-way construction relationship between 2D perspective and 3D models. [Copyright &y& Elsevier]

Abstract: The popularity of augmented reality (AR) applications on mobile devices is increasing, but there is as yet little research on their use in real-settings. We review data from two pioneering field trials where MapLens, a magic lens that augments paper-based city maps, was used in small-group collaborative tasks. The first study compared MapLens to a digital version akin to Google Maps, the second looked at using one shared mobile device vs. using multiple devices. The studies find place-making and use of artefacts to communicate and establish common ground as predominant modes of interaction in AR-mediated collaboration with users working on tasks together despite not needing to. [Copyright &y& Elsevier]

A variety of different approaches have been used to add digital ink annotations to text-based documents. While the majority of research in this field has focused on annotation support for static documents, a small number of studies have investigated support for documents in which the underlying content is changed. Although the approaches used to annotate static documents have been relatively successful, the annotation of dynamic text documents poses significant challenges which remain largely unsolved. However, it is difficult to clearly identify the successful techniques and the remaining challenges since there has not yet been a comprehensive review of digital ink annotation research. This paper reports the results of a systematic mapping study of existing work, and presents a taxonomy categorizing digital ink annotation research. [ABSTRACT FROM AUTHOR]

A comprehensive study of the Earth System and its different environments requires understanding of multi-dimensional data acquired with a multitude of different sensors or produced by various models. Here we present a component-wise scalable web-based framework for simultaneous visualisation of multiple data sources. It helps contextualise mixed observation and simulation data in time and space. This work is an extended version of the conference paper (Buck et al., 2021). [Display omitted] • Open-source (EUPL) hybrid application for realtime visualisation of 4D geoscientific data. • Split into native server and web client to utilize the strengths of both platforms. • Desktop builds are released for Windows, Linux, and MacOS. • Viewer used on expedition cruises to plan underwater exploration missions • Presentation and data validation capabilities used by GLODAP [ABSTRACT FROM AUTHOR]

This paper proposes a methodology for computing, visualizing, and analyzing critical decision boundaries for the selection of shortest paths in a given environment. Decision boundaries are defined as the points in a map from which two or more different shortest paths exist towards a destination. This paper introduces the problem of visualizing their evolution, taking into account moving obstacles, moving goals, and as well multiple goals. The proposed visualizations enable analyzing which paths should be taken and at which departure times, such that a destination can be reached by the shortest possible path when taking into account a moving target or time-varying areas to be avoided. The proposed techniques are also applied to the analysis and improvement of exit placement in a given environment, in order to improve the evacuation flow in emergency situations. [Display omitted] • This research presents a unique method for detecting decision boundaries in a given environment, based on the analysis of the generator points of the Shortest Path Map (SPM) rather than employing traditional scalar field topological methods relying on cell neighborhood information which can be affected by the representation resolution. • The proposed approach introduces tools and techniques to visualize the evolution of decision boundaries when considering dynamically-changing obstacles and targets, and to design exit placement to equalize the escape flow distribution. • This novel approach supports decision-making applications related to navigation and environment modeling in emergency evacuation planning. • By analyzing and visualizing SPM decision boundaries, the lengths of globally-optimal Euclidean shortest paths are taken into account, instead of grid-based accumulated distances used in other approaches. [ABSTRACT FROM AUTHOR]

Image deraining aims to remove rain streaks from images and reduce information loss in outdoor images caused by rain. As a fundamental task in image processing, image deraining not only enhances the visibility of images but also provides necessary image restoration for advanced vision tasks. Existing image deraining models mostly train end-to-end models by minimizing the similarity between the output image of the model and the rain-free ground truth. Although these methods have achieved significant results, they often perform poorly in the face of dense and changing rain streak scenes. In this paper, we propose a novel method, called D ual-Channel C omponent D ecomposition Net work (DCD-Net). The basic idea of DCD-Net is to leverage the separability prior of rainy images, treats the rain-free background layer and the rain streak mask layer as two parallel component extraction tasks. To this end, it builds a dual-branch parallel networks that extract the rain-free background image and decouple the reconstruction information of the rain streak mask, respectively. It finally applies a composite multi-level contrastive supervision to the output of the above dual-branch parallel network, thereby achieving rain streak removal. Extensive experiments on various datasets demonstrate that the proposed model outperforms existing methods in deraining dense rain streak images. [Display omitted] • This paper proposes an image deraining method, called Dual-Channel Component Decomposition Network (DCD-Net). • DCD-Net treats the rain-free background layer and the rain streak mask layer as two parallel component extraction tasks. • DCD-Net obtains competitive performance in deraining complex and dense rain streak images. [ABSTRACT FROM AUTHOR]

Image-to-Image (I2I) translation methods based on generative adversarial networks (GANs) require large amounts of training data, without which they will suffer from over-fitting and train divergence, and trained models are sub-optimal. In addition, it would be very difficult for the model to synthesize high-frequency signals, deteriorating the synthesis quality. To address these, this paper proposes W 2 GAN, which mainly introduces the ideas of the Importance Weight and Wavelet transformation to achieve the I2I translation trained on limited-data. Concretely, this paper first alleviates the over-fitting and train divergence by the adversarial loss with importance weight, which aims to improve the influence of the high-quality generated images during the training generator, thus enhancing the generator to deceive the discriminator. Then, the high-frequency features of the wavelet transformation are applied to the decoder, and wavelet-AdaIN normalization is proposed to prevent deficiency of high-frequency information, which adaptively integrates high-frequency statistical characteristics from generated features and real image high-frequency information. Qualitative and quantitative results on the AFHQ and CelebA-HQ datasets demonstrate the merits of the W 2 GAN. Noticeably, this paper achieves state-of-the-art FID and KID on AFHQ and CelebA-HQ datasets. [Display omitted] • In this paper, the GANs are trained under limited data. This paper overcomes the problem of the discriminator over-fitting during the training, which leads to the model divergence and the results degradation, stabilizing the training process and achieving higher quality results. From the qualitative and quantitative perspectives, this paper has achieved competitive results in current research. • This paper introduces the learnable importance weight to the adversarial loss, which aims to hope the high-quality images produce higher influence during the training generator. It relieves the problems of the training diverge and over-fitting. • This paper proposes a Wavelet-AdaIN Normalization to learn the high-frequency features, which adaptively integrates high-frequency statistical characteristics from generated features and real image high-frequency information. It encourages the generator to produce precise high-frequency signals with fine details. [ABSTRACT FROM AUTHOR]

Real-time semantic segmentation, as a key technique for scene understanding, has been an important research topic in the field of computer vision in recent years. However, existing models are unable to achieve good segmentation accuracy on mobile devices due to their huge computational overhead, which makes it difficult to meet actual industrial requirements. To address the problems faced by current semantic segmentation tasks, this paper proposes a lightweight bilateral asymmetric residual network (LBARNet) for real-time semantic segmentation. First, we propose the bilateral asymmetric residual (BAR) module. This module learns multi-scale feature representations with strong semantic information at different stages of the semantic information extraction branch, thus improving pixel classification performance. Secondly, the spatial information extraction (SIE) module is constructed in the spatial detail extraction branch to capture multi-level local features of the shallow network to compensate for the lost geometric information in the downsampling stage. At the same time, we design the attention mechanism perception (AMP) module in the jump connection part to enhance the contextual representation. Finally, we design the dual branch feature fusion (DBF) module to exploit the correspondence between higher-order features and lower-order features to fuse spatial and semantic information appropriately. The experimental results show that LBARNet, without any pre-training and pre-processing and using only 0.6M parameters, achieves 70.8% mloU and 67.2% mloU on the Cityscapes dataset and Camvid dataset, respectively. LBARNet maintain a high segmentation accuracy while using a smaller number of parameters compared to most existing state-of-the-art models. [Display omitted] • This paper proposes a Bilateral Asymmetric Residual (BAR) module, a Spatial Information Extraction (SIE) module, an Attention Mechanism Perception (AMP) module and a Dual Branch Feature Fusion (DBF) module. • A Lightweight Bilateral Asymmetric Residual Network (LBARNet) for real-time image semantic segmentation is proposed in this article. • The experimental results show that LBARNet achieves 70.8% and 67.2% segmentation accuracy on two challenging datasets (Cityscapes and CamVid) using only 0.6M parameters. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a general concept for the analysis of temporal and multivariate data and the system LFPeers that applies this concept to temporal similarity search and results exploration. The conceptual workflow divides the analysis in two phases: a search phase to find the most similar objects to a query object before a time point t 0 in the temporal data, and an exploration phase to analyze and contextualize this subset of objects after t 0. LFPeers enables users to search for peers through interactive similarity search and filtering, explore interesting behavior of this peer group, and learn from peers through the assessment of diverging behaviors. We present the conceptual workflow to learn from peers and the LFPeers system with novel interfaces for search and exploration in temporal and multivariate data. An earlier workshop publication for LFPeers included a usage scenario targeting epidemiologists and the public who want to learn from the Covid-19 pandemic and distinguish successful and ineffective measures. In this extended paper, we now show how our concept is generalized and applied by domain experts in two case studies, including a novel case on stocks data. Finally, we reflect on the new state of development and on the insights gained by the experts in the case studies on the search and exploration of temporal data to learn from peers. [Display omitted] • Conceptual framework for temporal and multimodal similarity search and exploration. • Visual Analytics system for cause–effect analysis on temporal data. • Interactive user-defined search and exploration of data objects by similarity. [ABSTRACT FROM AUTHOR]

The paper presents a novel solution to the issue of incomplete regions in 3D meshes obtained through digitization. Traditional methods for estimating the surface of missing geometry and topology often yield unrealistic outcomes for intricate surfaces. To overcome this limitation, the paper proposes a neural network-based approach that generates points in areas where geometric information is lacking. The method employs 2D inpainting techniques on color images obtained from the original mesh parameterization and curvature values. The network used in this approach can reconstruct the curvature image, which then serves as a reference for generating a polygonal surface that closely resembles the predicted one. The paper's experiments show that the proposed method effectively fills complex holes in 3D surfaces with a high degree of naturalness and detail. This paper improves the previous work in terms of a more in-depth explanation of the different stages of the approach as well as an extended results section with exhaustive experiments. [Display omitted] • We perform 3D surface reconstructions using generative inpainting techniques. • 2D representation of a 3D surface geometry based on its curvature. • Application of a general purpose neural network for inpainting. • Our approach does not require dataset nor training time. • Results outperform state-of-the-art quality and naturalness of the reconstructions. [ABSTRACT FROM AUTHOR]

Open-set semantic segmentation remains yet a challenging task, not only due to the inherent challenges of pixel-wise classification but also the precise segmentation of categories not seen during training. The pursuit of that task is rapidly growing in the Computer Vision community, urging the need to organize the literature. In this paper, we extend our previous work by conducting a more comprehensive systematic mapping of the open-set segmentation literature between January 2001 and January 2023 and proposing a novel taxonomy. Our goal is to provide a broad understanding of current trends for the open-set semantic segmentation (OSS) task defined by existing approaches that may influence future methods. By characterizing methodologies in terms of open-set identification strategies, data inputs, and other relevant aspects, we present a structured view of how researchers are advancing in the field of open-set semantic segmentation. To the best of the authors' knowledge, this is the first systematic review of OSS methods. Moreover, we apply the proposed taxonomy to selected methods for open-set recognition, outlining important similarities and differences of such a closely related field. [Display omitted] • Systematic review of papers related to open-set semantic segmentation for the past 20 years. • The proposed taxonomy aims to organize the literature on open-set segmentation. • Seminal papers on open-set recognition are classified under the proposed taxonomy. • Applications like autonomous driving and remote sensing were found to commonly resort to the open set strategy. • Methods tackling open-world are becoming more commom; [ABSTRACT FROM AUTHOR]

New technology such as virtual reality can help computer graphics education, for example, by providing the opportunity to illustrate challenging 3D procedures. RePiX VR is a virtual reality tool for computer graphics education that focuses on teaching the core ideas of the rendering pipeline. This paper describes the development and two initial evaluations, which aimed to strengthen the usability, review requirements for different stakeholders, and build infrastructure for learning analytics and research. The integration of learning analytics raises the question of appropriate indicators to be approached through exploratory data analysis. In addition to learning analytics, the evaluation includes quantitative techniques to get insights about usability, and didactical feedback. This paper discusses advanced aspects of learning in VR and looks specifically at movement behavior. According to the evaluations, even learners without prior experience can utilize the VR tool to pick up the fundamentals of computer graphics. [Display omitted] • Evaluated educational VR environment for teaching Computer Graphics. • Teaching Computer Graphics in Virtual Reality is promising. • Learners have various movement and teleportation pattern and different interaction behavior. • A comparison of Desktop and VR users shows differences between groups, as well as a comparison of novices and experts. • The evaluation contains Multimodal Learning Analytics, Quantitative Feedback, and Usability aspects. [ABSTRACT FROM AUTHOR]

In this paper, the non-uniform illumination enhancement problem of underwater images under the artificial light sources conditions is investigated based on Convolution Neural Network (CNN). First, we propose a trainable end-to-end enhancer called NUIENet, for enhancing the non-uniform illumination of underwater images. The proposed model consists of correction network and fusion layers. The correction network adopts the encoder–decoder structure with skip connections to enhance the features of different channels in the HSV domain, and then these enhanced features are fused by the fusion layers to obtain the desired high-quality images. Second, we built an underwater images dataset using Generative Adversarial Network (GAN) and Gaussian Function. Finally, both qualitative and quantitative experimental results show that the proposed method can produce better performance compared to other state-of-the-art enhancement methods on both real-word and synthetic underwater dataset. • This paper proposes a non-uniform illumination enhancer CNN-based which uses the encoder–decoder structure with skip connections to enhance the underwater images with NUI to the desired high-quality images. • To boost underwater imaging processing, we construct a dataset of the underwater image with NUI based on the GAN and Gaussian function which contains NUI images and their corresponding high-quality reference image. • Compared with other state-of-the-art NUIE methods, the proposed network achieves a nature color correction and superior or equivalent visibility improvement. [Display omitted] [ABSTRACT FROM AUTHOR]

Depth estimation is crucial for scene understanding and downstream tasks, especially the self-supervised training methods showing great potential. The overall structure and local details of the scene are essential for improving the quality of depth estimation. The proposal of Monodepth2 has led to significant progress in self-supervised monocular depth estimation. However, Monodepth2 uses the most basic encoder–decoder architecture. The limited data flow information of the network leads to a large semantic gap between the encoder and the decoder, which reduces the accuracy of the network for fine-grained feature recognition. Monodepth2 adopts Resnet18 pre-trained on the Imagenet dataset as the encoder. This traditional convolutional pooling structure results in a loss of pixel information in the network at every scale. In order to solve this problem, this paper proposes an improved DepthNet. The network adopts Hrnet in semantic segmentation as the base encoder, which adopts an advanced multi-scale fusion method in the whole process, thus avoiding the loss of pixel information. An additional densely connected U-Net is employed at the decoder side to provide more information flow. Furthermore, the semantic gap between the encoder and decoder is reduced by adding different numbers of residual connections and channel attention on each layer. The network structure can be regarded as a collection of fully convolutional networks. Since the deep features of the network have a higher correlation with the vertical position, we add a spatial location attention module to the deep-level network to reduce this semantic gap. The approach performs significantly well on the KITTI dataset benchmark, with several performance criteria comparable to supervised monocular depth inference methods. • This work is a deep estimation network for scene reconstruction and scene understanding. This network redesigns the self-supervised monocular depth framework from an entirely new perspective. The network uses HrNet from the field of semantic segmentation as the base encoder, which employs a progressive multi-scale fusion approach throughout, thus avoiding the loss of pixel information. • An additional densely connected U-Net is used at the decoder side to provide further information flow. To reduce the semantic gap between codecs, we add a different number of residual connections and channel attention on each layer. The network is not trained with the help of other auxiliary networks, and the performance of the depth estimation is improved only by stimulating the network's potential. • This work achieves best-in-class accuracy in monocular depth estimation. When the model in this paper is used for 3D scene reconstruction, it can perform a complete recovery of the scene structure. [ABSTRACT FROM AUTHOR]

Low-light image enhancement algorithms need recover overall information of images, including local details and global information. However, existing image enhancement methods mainly focus on local details or global information. Therefore, it is challenging to balance the two aspects at the same time. This paper proposes a local dual-branch network (BrightFormer) for image enhancement that combines convolutions and transformers as to solution. The salient features of this paper are: (1) convolution is adopted to refine high-frequency information so that local features are preserved and propagated throughout the network; (2) combining gated parameters with prior information on illumination (ill-map) in self-attention can not only improves the flexibility of feature expression but also extract global features more easily; (3) the obtained local details and global features are fused by spatial and channel attention in Feature equalization fusion unit (FEFU); (4) a Deep feedforward network (DFN) is utilized to encode the location information between adjacent pixels, and the GELU activation function is used to retain useful features and eliminate useless features with an attention-like mechanism. Experimental results show that BrightFormer achieves competitive performance on quantitative metrics and visual perception on the datasets such as LOL, MEF and LIME etc. [Display omitted] • Using cross-convolution can extract rich local features of the image. • Incorporates the gating mechanism and prior information on the illumination. • Different attention mechanisms are adopted for local and global features. • Use the GELU activation function as an attention mechanism. [ABSTRACT FROM AUTHOR]

Combustibles, topography, and weather factors are the three essential factors affecting forest fire behavior, and current forest fire spread models need to consider weather factors fully. This paper proposes a forest fire spread method based on environmental weather factors to present a visualized simulation of forest fire spread in the natural environment. Forest pyrolysis differs based on water content, so a single-tree pyrolysis model with temperature as its core has been constructed to describe the differences in forest pyrolysis during different seasons visually. In addition, based on the improved Huygens principle as the theoretical basis for forest fire spread, weather factors such as wind speed, wind direction, and precipitation controlled by weather are coupled with the forest fire spread process. And the forest fire spread in three-dimensional scenarios is simulated by considering environmental factors. The visualization of the forest fire extinguishing process caused by precipitation is realized. Finally, the interaction between rain and snow, terrain and trees is realized when precipitation affects the corresponding landscape and vegetation texture to enhance the realism of the constructed forest environment. In short, this paper proposes a forest fire spread method based on environmental weather factors, which intuitively expresses the influence of different weather factors on forest fire spread, thereby improving the immersive experience of the related senses and realizing realistic scene roaming. [Display omitted] • Based on the single tree pyrolysis model, differentiating forest burning. • Visualization of the influence of simulated weather factors on forest fire behavior. • Use texture mixing technology to construct 3D forest weather scene. [ABSTRACT FROM AUTHOR]

Visualizing the isosurfaces that represent material boundaries is an important technique for understanding the features of interest in a scalar volumetric dataset. However, one isosurface may contain multiple types of boundaries, i.e., boundaries between different pairs of materials. In this paper, we present a novel isosurface segmentation method that aids in learning structural information of a dataset by separating different types of boundaries in one isosurface. This method uses common boundary tests to classify a point on the isosurface. The test determines whether a point on the isosurface that is at a boundary shared by both the isosurface and a reference isosurface. It uses a gradient-guided sampling approach and is based on material boundary properties. A new region growing algorithm is developed to improve the segmentation results. Our new method can also be used to segment an isosurface that passes through both material boundaries and the interior of a material. Two applications of the new method are also demonstrated in the paper. One is to render and segment section planes to enhance visualization. The other one is to obtain more accurate and meaningful isosurface statistics. • Segment an isosurface that contains multiple types of material boundaries. • Use region growing approach to improve the isosurface segmentation results. • Correctly segment the section planes to show the interior structure of the object. • Segment an isosurface that passes through both material boundaries and material interior. • Obtain more accurate isosurface statistics by averaging them over different portions of the segmented isosurface. [Display omitted] [ABSTRACT FROM AUTHOR]

Growing public interest in understanding the origins and production methods of dairy products, driven by concerns related to environmental impact, local sourcing, and ethics, highlights an important trend. Nevertheless, a knowledge-trust gap persists between consumers and the dairy industry. Addressing this gap, in this paper, we developed an immersive virtual farm simulation to provide realistic on-farm experiences to the public. Within the virtual farm, users can explore various sites where dairy cows are raised and gain insights into dairy production processes using a head-mounted display (HMD). This simulation was demonstrated at local libraries, involving 48 public participants. We collected and analyzed participants' feedback on various aspects, including usability and their overall perceptions, to assess the simulation's effectiveness as an agricultural education tool. We investigated the impact of the virtual experience on participants' perceived knowledge gain and their awareness of the dairy industry. The results indicate that our dairy farm simulation was positively received as an effective tool for public education. Emphasizing the potential of virtual reality (VR) simulations in agricultural education and the industry, we discuss our key findings and future plans. [Display omitted] • Development process to build a realistic immersive simulation of a dairy farm for public education purposes. • Findings on the usability and user perception of the immersive experience for education purposes. • The system is an effective and useful tool for learning and the provision of information about the dairy industry. [ABSTRACT FROM AUTHOR]

For many applications, remote guidance and telerobotics provide great advantages. For example, tele-ultrasound can bring much-needed expert healthcare to isolated communities. However, existing tele-guidance methods have serious limitations including either low precision for video conference-based systems, or high complexity and cost for telerobotics. A new concept called human teleoperation leverages mixed reality, haptics, and high-speed communication to provide tele-guidance that gives an expert nearly-direct remote control without requiring a robot. This paper provides an overview of the human teleoperation concept and its application to tele-ultrasound. The concept and its impact are discussed. A new approach to remote streaming and control of point-of-care ultrasound systems independent of their manufacturer is described, as is a high-speed communication system for the HoloLens 2 that is compatible with ResearchMode API sensor stream access. Details of these systems are shown in supplementary video demonstrations. Novel interaction methods enabled by HoloLens 2-based pose tracking are also introduced and tests of the communication and user interaction are presented. The results show continued improvement of the system compared to previous work in instrumentation, HCI, and communication. The system thus has good potential for tele-ultrasound, as well as possible other applications of human teleoperation including remote maintenance, inspection, and training. The remote ultrasound streaming and control application is made available open source. [Display omitted] • System improvements to human teleoperation demonstrate its feasibility. • Other devices such as the Nreal Light can be used for implementation. • New device-agnostic remote ultrasound streaming and control demonstrated. • HoloLens pose tracking enables human teleoperation with limited compute resources. • HoloLens-based communication system provides effective sensor data streaming. [ABSTRACT FROM AUTHOR]

Learning 3D mesh representation is necessary for many computer vision and graphic tasks. Recently, some works have studied convolution methods for directly processing input meshes. However, these methods are usually weak in extracting local geometry information because of the disadvantages such as isotropic filter, neglect of mesh topology, and a small convolution field. In this paper, we introduce a local geometry-perceptive mesh convolution, which pays attention to mesh irregular structures for efficiently capturing geometry features in a multi-ring receptive field. Specifically, we define each template node's dynamic neighbor-attention weights used in multiple attention aggregation operations for obtaining local mesh change information of different vertices in the multi-ring field. After each aggregation, a shared anisotropic filter maps the catenation of each new vertex and its neighbors for extracting geometry features of the current ring. Then, complete local geometry features of each vertex in its large local field are obtained by summing the mapped results of each aggregation. Moreover, the position features of each vertex are added to its local geometry features to get the final representation vector of the vertex. We demonstrate the proposed mesh convolution method's strong ability in modeling 3D mesh shapes. [Display omitted] • Utilizing local irregular mesh structures, dynamic neighbor-attention weights of template's nodes perceive local mesh change information of different vertices. • Multiple attention aggregations and shared anisotropic filters help extract geometry features of large local receptive fields with multi-ring. • The proposed method determines our deep 3D morphable models (3DMMs) have smaller sizes than previous models and achieve state-of-the-art performances. [ABSTRACT FROM AUTHOR]

Point cloud completion aims to estimate the whole shapes of objects from their partial scans, and one of the main obstacles that prevents current methods from being applied in real-world scenarios is the variety of structural losses in real-scanned objects, which can hardly be fully included and reflected by the training samples. In this paper, we introduce Patch Sequence Autoregression Network (PSA-Net), a learning-based method that can be trained without the partial point clouds in dataset and is inherently adaptable to input scans with different levels of shape incompleteness: It makes restoring the unseen parts of objects be equivalent to predicting the missing tokens in local patch embedding sequences, and such prediction can start from any initial states. Specifically, we first introduce a Sequential Patch AutoEncoder that reconstructs complete point clouds from quantized patch feature sequences. Second, we establish a Mixed Patch Autoregression pipeline that can flexibly infer the whole sequence from any number of known tokens at any positions. Third, we propose a Voting-Based Mapping module that makes input points softly vote for their possible related tokens in sequences based on their local areas, which transforms partial point clouds to masked sequences in test. Quantitative and qualitative evaluations on two synthetic and four real-world datasets illustrate the competitive performances of our network when comparing with existing approaches. [Display omitted] • A Sequential Patch AutoEncoder for shape generation from quantized feature sequence. • A Mixed Patch Autoregression pipeline for token prediction from any initial states. • A Voting-based Mapping module for transformation from partial shapes to sequences. • Competitive performances on two synthetic and four real-world datasets. [ABSTRACT FROM AUTHOR]

Low-light images often suffer from severe visual degradation, affecting both human perception and high-level computer vision tasks. Most existing methods process images in the spatial domain, making it challenging to simultaneously improve brightness while suppressing noise. In this paper, we present a novel perspective to enhance images based on frequency domain characteristics. Specifically, we reveal that the low-frequency components are closely related to luminance and color, whereas the high-frequency components are not. Based on this observation, we propose the Frequency-aware Network (FaNet) for low-light image enhancement. By selectively adjusting low-frequency components, FaNet preserves more high-frequency details while achieving low-light image enhancement. Additionally, we employ a multi-scale framework and selective fusion for effective feature learning and image reconstruction. Experimental results demonstrate the superiority of the proposed method. [Display omitted] • We reveal that the luminance is closely related to low-frequency components. • We design a frequency-aware network to utilize frequency domain features. • A multi-scale framework and selective fusion is proposed for feature learning. [ABSTRACT FROM AUTHOR]

3D meshes are commonly utilized and may be considered to be the most popular surface representation in computer graphics due to their simplicity, efficiency and flexibility. However, the explicit storage of mesh vertices and connectivity, as in widely-used PLY and OBJ file formats, leads to substantial memory consumption. This, in turn, directly affects the processing and transmission in downstream applications. Though mesh simplification and mesh compression are common strategies to lessen memory consumption, they exhibit inherent limitations either in maintaining a balance between accuracy, efficiency, memory usage and mesh quality, or breaking the simplicity of explicit storage and struggling with optimizing the trade-off between compression performance and computational resource consumption. To overcome these limitations, inspired by the Gaussian Mixture Model (GMM), this paper proposes a GMM vertex-optimization-based implicitly-connected quadrilateral format for 3D mesh storage, named GMM-ICQ. Extensive qualitative and quantitative evaluations demonstrate that the GMM-ICQ format achieves efficient compression by retaining only a small amount of vertex information, while preserving sharp features and maintaining relatively high mesh quality. It also exhibits a certain degree of robustness in the presence of noise interference. Furthermore, benefiting from the inherent grid-based connectivity, the GMM-ICQ format maintains the simplicity of explicit storage and can be implemented as a progressive variant without incurring additional computational overhead. [Display omitted] • We present a GMM vertex-optimization-based implicitly-connected quadrilateral format for 3D mesh storage. • Simultaneously balances accuracy, efficiency, memory usage, and mesh quality. • Preserves the simplicity of explicit storage (such as PLY and OBJ). • No additional computational overhead needed for progressive variant implementation. [ABSTRACT FROM AUTHOR]

Typical non-local edge-aware filtering methods build long-range connections by deriving a minimum spanning tree (MST) from the input image. Each pixel on the MST only connects to a sub-set of pixels in the 8-connected neighborhood, resulting in piece-wise constant output with fake edges among sub-trees for the unbalanced information propagation along eight directions. In this paper, we propose two complementary spatial trees to incorporate information from the entire image. The structure of each tree depends on the spatial relationships of neighboring pixels. The distances between any two pixels in both spatial space and intensity space are the shortest distances on each tree. We introduce an efficient algorithm to recursively compute the output and the normalization constant on each tree with linear time complexity. For each pixel, we first calculate the outputs from eight subtrees and then fuse them to obtain the result on each tree structure. The final filtering output of our method is the weighted average of the results from two complementary spatial trees. Moreover, we present a distance mapping scheme to adjust the intensity distance between neighboring pixels, enabling our method to filter out a manageable degree of low-amplitude structures while sharpening major edges. Extensive experiments in graphic applications, such as image denoising, JPEG artifact removal, tone mapping, detail enhancement, and colorization, demonstrate the effectiveness and versatility of our method. [Display omitted] • Novel complementary trees to estimate the geodesic distance between any two pixels. • Efficient algorithms with linear time complexity to compute the weighted average of all pixels in the input image. • A distance mapping scheme in intensity space to manageably filter out low-amplitude structures. • Quantitative evaluation and qualitative comparison on various graphic applications to show the effectiveness and the versatility of our approach. [ABSTRACT FROM AUTHOR]

Image animation creates visually compelling effects by animating still source images according to driving videos. Recent work performs animation on arbitrary objects using unsupervised methods and can relatively robustly perform motion transfer on human bodies. However, the complex representation of motion and unknown correspondence between human bodies often lead to issues such as distorted limbs and missing semantics, which make human animation challenging. In this paper, we propose a semantically guided, unsupervised method of motion transfer, which uses semantic information to model motion and appearance. Specifically, we use a pre-trained human parsing network to encode the rich and diverse foreground semantic information, thus generating fine details. Secondly, we use a cross-modal attention layer to learn the semantic region's correspondence between human bodies to guide the network in selecting appropriate input features, prompting the network to generate accurate results. Experiments demonstrate that our method outperforms state-of-the-art methods in motion-related metrics, while effectively addressing the problems of semantic missing and unclear limb structures prevalent in human motion transfer. These improvements can facilitate its applications in various fields, such as education and entertainment. [Display omitted] • Proposes a novel framework for human image animation using semantic features and cross-modal attention. • Introduces semantic segmentation features to represent motion and identity information. • Employs a cross-modal attention mechanism to establish correspondences between semantic regions. • Achieves state-of-the-art performance on complex human motions like TaiChi poses. • Reduces issues of missing semantics and distorted limbs common in human animation. [ABSTRACT FROM AUTHOR]

In recent times, there has been notable and swift advancement in the field of image dehazing. Several deep learning techniques have demonstrated remarkable proficiency in resolving homogeneous dehazing issues. Nonetheless, the current dehazing approaches are generally formulated to deal with homogeneous haze, which is often undermined in real-world scenarios due to the uncertain haze dispersion. In this paper, we propose a dehazing model named RepDehazeNet by combining a structurally Reparameterization Encoder-Decoder subnet and a Full-Resolution Attention subnet. To be specific, the structural reparameterization idea is introduced into the encoder–decoder subnet to strengthen the feature extraction of dehazed images and improve the feature extraction speed. RepDehazeNet is compared with seven SOTA models on different datasets in terms of PSNR, SSIM, parameter quantity, and inference time. Compared to the DW-GAN model, the proposed RepDehazeNet model reduces the number of parameters by 2.7 million, and improves the inference speed by 90.3%, while achieving a higher PSNR of 0.5 dB on the NH-Haze2021 dataset. The experimental results demonstrate that the proposed RepDehazeNet model can effectively improve the real-time performance, accuracy of dehazing synthesized and nonhomogeneous haze images. [Display omitted] • Structural reparameterization dehazenet: outstanding performance, faster speed. • Replacing Tanh with ReLU leads to better results. • Transfer learning addresses the problem of insufficient samples. • Dual subnets method proves highly effective in datasets of different scales. [ABSTRACT FROM AUTHOR]

Currently, deep learning-based 3D face reconstruction methods have shown promising results. However, they ignore the contextual information of the face, which is a topologically unified entirety. This paper proposes a 3D face reconstruction approach based on hybrid-level contextual information. Firstly, we suggest a regression network with contextual modeling capability at the feature level, PPR-CNet, which adopts a preferential parameter regression to regress the 3DMM parameters dynamically based on their various impacts on the reconstructed 3D face. Furthermore, we design a contextual landmark loss to constrain the face geometry at the landmark level. We introduce a differentiable renderer combined with multiple loss functions for weakly-supervised training. Quantitative experiments on two benchmarks show our method outperforms several SOTA methods. Extensive qualitative experiments indicate that our method performs efficiently in realism, facial proportion, and occlusion. [Display omitted] • We propose an approach to reconstruct a 3D face from a single image based on hybrid-level contextual information. • We propose a regression network, PPR-CNet, with contextual modeling capability, which regresses 3DMM parameters dynamically. • We design a contextual landmark loss to constrain face geometry employing landmark contextual information. [ABSTRACT FROM AUTHOR]

This paper proposes a novel curve called the α - curve , which is specified by a variable shape parameter α and can be applied in curve reconstruction involving curve fairing, fitting, and segmentation. In contrast to classical log-aesthetic curves in which α is constant, this study contributes a novel formulation of α that varies monotonically within a specified segment. By introducing the variable α , the logarithmic curvature graph (LCG) becomes piecewise linear and parabolic, or any other function if necessary. Consequently, in comparison to conventional log-aesthetic curves that have limited field usage based on their rigidity, α -curves have various LCGs and flexible representations, thereby opening up many practical applications. Experimental results and comparative studies on curves created by CAD experts in the automotive industry demonstrate the theoretical and practical validity of α -curves. [Display omitted] • α -Curves are proposed by extending the mathematical framework of log-aesthetic curves • An α -curve is specified by a variable shape parameter • α -Curves have various log-curvature graphs (LCG) and flexible representations • α -Curves are applied to a new fairing, avoiding conventional issues of side effects [ABSTRACT FROM AUTHOR]