Search

AbstractBackgroundObjectivesMaterials and methodsResultsConclusions and significanceOptimizing the educational experience of trainees in the operating room is important; however, ear anatomy and otologic surgery are challenging for trainees to grasp. Viewing otologic surgeries often involves limitations related to video quality, such as visual disturbances and instability.We aimed to (1) improve the quality of surgical videos (tympanomastoidectomy [TM]) by using artificial intelligence (AI) techniques and (2) evaluate the effectiveness of processed videos through a questionnaire-based assessment from trainees.We conducted prospective study using video inpainting and stabilization techniques processed by AI. In each study set, we enrolled 21 trainees and asked them to watch processed videos and complete a questionnaire.Surgical videos with the video inpainting technique using the implicit neural representation (INR) model were found to be the most helpful for medical students (0.79 ± 0.58) in identifying bleeding focus. Videos with the stabilization technique via point feature matching were more helpful for low-grade residents (0.91 ± 0.12) and medical students (0.78 ± 0.35) in enhancing overall visibility and understanding surgical procedures.Surgical videos using video inpainting and stabilization techniques with AI were beneficial for educating trainees, especially participants with less anatomical knowledge and surgical experience. [ABSTRACT FROM AUTHOR]

We present a novel and flexible learning-based method for generating tileable image sets. Our method goes beyond simple self-tiling, supporting sets of mutually tileable images that exhibit a high degree of diversity. To promote diversity we decouple structure from content by foregoing explicit copying of patches from an exemplar image. Instead we leverage the prior knowledge of natural images and textures embedded in large-scale pretrained diffusion models to guide tile generation constrained by exterior boundary conditions and a text prompt to specify the content. By carefully designing and selecting the exterior boundary conditions, we can reformulate the tile generation process as an inpainting problem, allowing us to directly employ existing diffusion-based inpainting models without the need to retrain a model on a custom training set. We demonstrate the flexibility and efficacy of our content-aware tile generation method on different tiling schemes, such as Wang tiles, from only a text prompt. Furthermore, we introduce a novel Dual Wang tiling scheme that provides greater texture continuity and diversity than existing Wang tile variants. [ABSTRACT FROM AUTHOR]

Crafting a single, versatile physics-based controller that can breathe life into interactive characters across a wide spectrum of scenarios represents an exciting frontier in character animation. An ideal controller should support diverse control modalities, such as sparse target keyframes, text instructions, and scene information. While previous works have proposed physically simulated, scene-aware control models, these systems have predominantly focused on developing controllers that each specializes in a narrow set of tasks and control modalities. This work presents MaskedMimic, a novel approach that formulates physics-based character control as a general motion inpainting problem. Our key insight is to train a single unified model to synthesize motions from partial (masked) motion descriptions, such as masked keyframes, objects, text descriptions, or any combination thereof. This is achieved by leveraging motion tracking data and designing a scalable training method that can effectively utilize diverse motion descriptions to produce coherent animations. Through this process, our approach learns a physics-based controller that provides an intuitive control interface without requiring tedious reward engineering for all behaviors of interest. The resulting controller supports a wide range of control modalities and enables seamless transitions between disparate tasks. By unifying character control through motion inpainting, MaskedMimic creates versatile virtual characters. These characters can dynamically adapt to complex scenes and compose diverse motions on demand, enabling more interactive and immersive experiences. [ABSTRACT FROM AUTHOR]

While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for testtime optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. The code is available at https://github.com/CVMI-Lab/TEXGen. [ABSTRACT FROM AUTHOR]

The majority of existing face inpainting methods primarily focus on generating a single result that visually resembles the original image. The generation of diverse and plausible results has emerged as a new branch in image restoration, often referred to as "Pluralistic Image Completion". However, most diversity methods simply use random latent vectors to generate multiple results, leading to uncontrollable outcomes. To overcome these limitations, we introduce a novel architecture known as the Reference-Guided Directional Diverse Face Inpainting Network. In this paper, instead of using a background image as reference, which is typically used in image restoration, we have used a face image, which can have many different characteristics from the original image, including but not limited to gender and age, to serve as a reference face style. Our network firstly infers the semantic information of the masked face, i.e., the face parsing map, based on the partial image and its mask, which subsequently guides and constrains directional diverse generator network. The network will learn the distribution of face images from different domains in a low-dimensional manifold space. To validate our method, we conducted extensive experiments on the CelebAMask-HQ dataset. Our method not only produces high-quality oriented diverse results but also complements the images with the style of the reference face image. Additionally, our diverse results maintain correct facial feature distribution and sizes, rather than being random. Our network has achieved SOTA results in face diverse inpainting when writing. Code will is available at https://github.com/nothingwithyou/RDFINet. [ABSTRACT FROM AUTHOR]

In response to the inadequate utilization of prior information in current mural inpainting processes, leading to issues such as semantically unreliable inpaintings and the presence of artifacts in the inpainting area, a Dunhuang mural inpainting method based on reference guidance and multi‐scale feature fusion is proposed. First, the simulated broken mural, the mask image, and the reference mural are input into the model to complete the multi‐level embedding of patches and align the multi‐scale fine‐grained features of damaged murals and reference murals. Following the patch embedding module, a hybrid residual module is added based on hybrid attention to fully extract mural features. In addition, by continuing the residual concatenation of outputs of the hierarchical embedding module improves the ability of the model to represent deeper features, and improves the robustness and generalisation of the model. Second, the encoded features are fed into the decoder to generate decoded features. Finally, the convolutional tail is employed to propagate them and complete the mural painting. Experimental validation on the Dunhuang mural dataset demonstrates that, compared to other algorithms, this model exhibits higher evaluation metrics in the inpainting of extensively damaged murals and demonstrates overall robustness. In terms of visual effects, the results of this model in the inpainting process exhibit finer textures, richer semantic information, more coherent edge structures, and a closer resemblance to authentic murals. [ABSTRACT FROM AUTHOR]

With the rapid development of stereoscopic display technology, how to generate high-quality virtual view images has become the key in the applications of 3D video, 3D TV and virtual reality. The traditional virtual view rendering technology maps the reference view into the virtual view by means of 3D transformation, but when the background area is occluded by the foreground object, the content of the occluded area cannot be inferred. To solve this problem, we propose a virtual view acquisition technique for complex scenes of monocular images based on a layered depth image (LDI). Firstly, the depth discontinuities of the edge of the occluded area are reasonably grouped by using the multilayer representation of the LDI, and the depth edge of the occluded area is inpainted by the edge inpainting network. Then, the generative adversarial network (GAN) is used to fill the information of color and depth in the occluded area, and the inpainting virtual view is generated. Finally, GAN is used to optimize the color and depth of the virtual view, and the high-quality virtual view is generated. The effectiveness of the proposed method is proved by experiments, and it is also applicable to complex scenes. [ABSTRACT FROM AUTHOR]

Traditional dental prosthetics require a significant amount of work, labor, and time. To simplify the process, a method to convert teeth scan images, scanned using an intraoral scanner, into 3D images for design was developed. Furthermore, several studies have used deep learning to automate dental prosthetic processes. Tooth images are required to train deep learning models, but they are difficult to use in research because they contain personal patient information. Therefore, we propose a method for generating virtual tooth images using image-to-image translation (pix2pix) and contextual reconstruction fill (CR-Fill). Various virtual images can be generated using pix2pix, and the images are used as training images for CR-Fill to compare the real image with the virtual image to ensure that the teeth are well-shaped and meaningful. The experimental results demonstrate that the images generated by the proposed method are similar to actual images. In addition, only using virtual images as training data did not perform well; however, using both real and virtual images as training data yielded nearly identical results to using only real images as training data. [ABSTRACT FROM AUTHOR]

This study presents a new image inpainting model based on U-Net and incorporating the Wasserstein Generative Adversarial Network (WGAN). The model uses skip connections to connect every encoder block to the corresponding decoder block, resulting in a strictly symmetrical architecture referred to as Symmetric Connected U-Net (SC-Unet). By combining SC-Unet with a GAN, the study aims to reconstruct images more effectively and seamlessly. The traditional discriminators only differentiate the entire image as true or false. In this study, the discriminator calculated the probability of each pixel belonging to the hole and non-hole regions, which provided the generator with more gradient loss information for image inpainting. Additionally, every block of SC-Unet incorporated a Dilated Convolutional Neural Network (DCNN) to increase the receptive field of the convolutional layers. Our model also integrated Multi-Head Self-Attention (MHSA) into selected blocks to enable it to efficiently search the entire image for suitable content to fill the missing areas. This study adopts the publicly available datasets CelebA-HQ and ImageNet for evaluation. Our proposed algorithm demonstrates a 10% improvement in PSNR and a 2.94% improvement in SSIM compared to existing representative image inpainting methods in the experiment. [ABSTRACT FROM AUTHOR]

The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model‐based and data‐driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log‐likelihood of the patch distribution, and penalizing Wasserstein‐like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super‐resolution, and inpainting. Indeed, the approach provides also high‐quality results in zero‐shot super‐resolution, where only a low‐resolution image is available. The article is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance. [ABSTRACT FROM AUTHOR]

The rapid advancement of image inpainting tools, especially those aimed at removing artifacts, has made digital image manipulation alarmingly accessible. This paper proposes several innovative ideas for detecting inpainting forgeries based on a low-level noise analysis by combining Dual-Tree Complex Wavelet Transform (DT-CWT) for feature extraction with convolutional neural networks (CNN) for forged area detection and localization, and lastly by employing an innovative combination of texture segmentation with noise variance estimations. The DT-CWT offers significant advantages due to its shift-invariance, enhancing its robustness against subtle manipulations during the inpainting process. Furthermore, its directional selectivity allows for the detection of subtle artifacts introduced by inpainting within specific frequency bands and orientations. Various neural network architectures were evaluated and proposed. Lastly, we propose a fusion detection module that combines texture analysis with noise variance estimation to give the forged area. Also, to address the limitations of existing inpainting datasets, particularly their lack of clear separation between inpainted regions and removed objects—which can inadvertently favor detection—we introduced a new dataset named the Real Inpainting Detection Dataset. Our approach was benchmarked against state-of-the-art methods and demonstrated superior performance over all cited alternatives. [ABSTRACT FROM AUTHOR]

The issue of distortion or misalignment in 3D modeling resulting from moving objects such as people and vehicles during UAV-obtained oblique image acquisition was addressed. An enhanced GAN-based image restoration technique was proposed, which modifies the GAN network by incorporating a U-Net architecture in the generator, fortified with a dual-channel attention mechanism via CBAM in the connecting layers, thereby enhancing the technique's capability for restoring local image details. The discriminator was augmented with the VGG16 model and the SE-Net channel attention mechanism has been undertaken to ensure the high fidelity of generated images in the present approach. Image analysis and processing were carried out using Context Capture software, facilitating automatic 3D modeling. This methodology enables proactive removal of moving entities, such as people and vehicles, from high resolution, extensive oblique imagery, thus minimizing their detrimental effects on subsequent 3D modeling and enhancing model accuracy. The presented algorithm demonstrates superior performance compared to conventional GAN and WGAN-GP networks, exhibiting increases of 3. 329 96 dB and 0. 097 9 in PSNR values and 2. 288 94 dB and 0. 047 8 in SSIM indices, respectively. Moreover, through comparison with generated 3D models, the method effectively reduces geometric deformation and road texture mapping errors, leading to heightened model precision. [ABSTRACT FROM AUTHOR]

Many large-scale regularized inverse problems in imaging such as image restoration and reconstruction can be modeled as a generic objective function involves sum of nonsmooth but proximable terms, which are usually linear-operator-coupled. For the solution of these problems, a parallel linearized alternating direction method of multipliers (PLADMM) is proposed in this paper. At each step of the proposed algorithm, the proximity operators of the nondifferential terms are called individually. This leads to a highly parallel algorithm structure, where most sub-steps can be simultaneously solved. Profiting from the linearization step, the linear inverse operation is excluded. The convergence property of the proposed method is analyzed. The image deblurring, inpainting, and pMRI reconstruction experiments show that the proposed method has vast applicable vistas. Compared with the state-of-the-art methods, such as PADMM [21], FTVD-v4 [22], PPDS [33], FUSL [8], LADM [36], and ALADMM [27], it gains competitive results both in terms of quantitative indicators, such as PSNR or SSIM, and in terms of visual impression. [ABSTRACT FROM AUTHOR]

Visual aesthetics has always been an important area of computational vision, and researchers have continued exploring it. To further improve the performance of the image aesthetic evaluation task, we introduce a Transformer into the image aesthetic evaluation task. This paper pioneers a novel self-supervised image aesthetic evaluation model founded upon Transformers. Meanwhile, we expand the pretext task to capture rich visual representations, adding a branch for inpainting the masked images in parallel with the tasks related to aesthetic quality degradation operations. Our model’s refinement employs the innovative uncertainty weighting method, seamlessly amalgamating three distinct losses into a unified objective. On the AVA dataset, our approach surpasses the efficacy of prevailing self-supervised image aesthetic assessment methods. Remarkably, we attain results approaching those of supervised methods, even while operating with a limited dataset. On the AADB dataset, our approach improves the aesthetic binary classification accuracy by roughly 16% compared to other self-supervised image aesthetic assessment methods and improves the prediction of aesthetic attributes. [ABSTRACT FROM AUTHOR]

To enhance the appeal of residential real estate listings and captivate online customers, clean and visually convincing indoor scenes are highly desirable. In this research, we introduce an innovative image inpainting model designed to seamlessly replace undesirable elements within images of indoor residential spaces with realistic and coherent alternatives. While Generative Adversarial Networks (GANs) have demonstrated remarkable potential for removing unwanted objects, they can be resource-intensive and face difficulties in consistently producing high-quality outcomes, particularly when unwanted objects are scattered throughout the images. To empower small- and medium-sized businesses with a competitive edge, we present a novel GAN model that is resource-efficient and requires minimal training time using arbitrary mask generation and a novel half-perceptual loss function. Our GAN model achieves compelling results in removing unwanted elements from indoor scenes, demonstrating the capability to train within a single day using a single GPU, all while minimizing the need for extensive post-processing. [ABSTRACT FROM AUTHOR]

The authors propose Point'n Move, a method that achieves interactive scene object manipulation with exposed region inpainting. Interactivity here further comes from intuitive object selection and real‐time editing. To achieve this, Gaussian Splatting Radiance Field is adopted as the scene representation and its explicit nature and speed advantage are fully leveraged. Its explicit representation formulation allows to devise a 2D prompt points to 3D masks dual‐stage self‐prompting segmentation algorithm, perform mask refinement and merging, minimize changes, and provide good initialization for scene inpainting and perform editing in real‐time without per‐editing training; all lead to superior quality and performance. The method was tested by editing both forward‐facing and 360 scenes. The method is also compared against existing methods, showing superior quality despite being more capable and having a speed advantage. [ABSTRACT FROM AUTHOR]

In recent years, deep learning models have dramatically influenced image inpainting. However, many existing studies still suffer from over‐smoothed or blurred textures when missing regions are large or contain rich visual details. To restore textures at a fine‐grained level, a multi‐stage inpainting approach is proposed, which applies a series of partial inpainting modules as well as a progressive inpainting module to inpaint missing areas from their boundaries to the centre successively. Some improvements are made on the partial convolutions to reduce artifacts like blurriness, which require a convolution kernel to contain known pixels more than a certain proportion. Towards photorealistic inpainting results, the intermediate outputs from each stage are used to compute the loss. Finally, to facilitate the training process, a multi‐step training is designed that progressively adds inpainting modules to optimize the model. Experiments show that this method outperforms the current excellent techniques on the publicly available datasets: CelebA, Places2 and Paris StreetView. [ABSTRACT FROM AUTHOR]

Since the release of AlphaFold, researchers have actively refined its predictions and attempted to integrate it into existing pipelines for determining protein structures. These efforts have introduced a number of functionalities and optimisations at the latest Critical Assessment of protein Structure Prediction edition (CASP15), resulting in a marked improvement in the prediction of multimeric protein structures. However, AlphaFold's capability of predicting large protein complexes is still limited and integrating experimental data in the prediction pipeline is not straightforward. In this study, we introduce AF_unmasked to overcome these limitations. Our results demonstrate that AF_unmasked can integrate experimental information to build larger or hard to predict protein assemblies with high confidence. The resulting predictions can help interpret and augment experimental data. This approach generates high quality (DockQ score > 0.8) structures even when little to no evolutionary information is available and imperfect experimental structures are used as a starting point. AF_unmasked is developed and optimised to fill incomplete experimental structures (structural inpainting), which may provide insights into protein dynamics. In summary, AF_unmasked provides an easy-to-use method that efficiently integrates experiments to predict large protein complexes more confidently. Integrating AlphaFold (AF) predictions with experimental data is not straightforward. Here, authors introduce AF_unmasked, a tool to integrate AF with experimental information to predict large or challenging protein assemblies with high confidence. [ABSTRACT FROM AUTHOR]

We consider a shape optimization based method for finding the best interpolation data in the compression of images with noise. The aim is to reconstruct missing regions by means of minimizing a data fitting term in an L p -norm, for 1 ⩽ p < + ∞, between original images and their reconstructed counterparts using linear diffusion PDE-based inpainting. Reformulating the problem as a constrained optimization over sets (shapes), we derive the topological asymptotic expansion of the considered shape functionals with respect to the insertion of small ball (a single pixel) using the adjoint method. Based on the achieved distributed topological shape derivatives, we propose a numerical approach to determine the optimal set and present numerical experiments showing the efficiency of our method. Numerical computations are presented that confirm the usefulness of our theoretical findings for PDE-based image compression. [ABSTRACT FROM AUTHOR]

• Language-based Subject Generation faces challenge in accurate portrayal of subject. • Nowadays Reference Guided Generation lacks ability to preserve subject identity. • Exemplar-based instructions with visual tokens preserve visual details of subject. • Model based guidance samples better quality images with different pose. • Our model achieved highest CLIP, DINO score and user study compared to others. This paper presents a novel approach to subject-driven image generation that addresses the limitations of traditional text-to-image diffusion models. Our method generates images using reference images without relying on language-based prompts. We introduce a visual detail preserving module that captures intricate details and textures, addressing overfitting issues associated with limited training samples. The model's performance is further enhanced through a modified classifier-free guidance technique and feature concatenation, enabling the natural positioning and harmonization of subjects within diverse scenes. Quantitative assessments using CLIP, DINO and Quality scores (QS), along with a user study, demonstrate the superior quality of our generated images. Our work highlights the potential of pre-trained models and visual patch embeddings in subject-driven editing, balancing diversity and fidelity in image generation tasks. Our implementation is available at https://github.com/8eomio/Subject-Inpainting. [Display omitted] To create your abstract, type over the instructions in the template box below. Fonts or abstract dimensions should not be changed or altered. [ABSTRACT FROM AUTHOR]

Recently, a regularized Kaczmarz method has been proposed to solve tensor recovery problems. In this article, we propose a sampling greedy average regularized Kaczmarz method. This method can be viewed as a block or mini‐batch version of the regularized Kaczmarz method, which is based on averaging several regularized Kaczmarz steps with a constant or adaptive extrapolated step size. Also, it is equipped with a sampling greedy strategy to select the working tensor slices from the sensing tensor. We prove that our new method converges linearly in expectation and show that the sampling greedy strategy can exhibit an accelerated convergence rate compared to the random sampling strategy. Numerical experiments are carried out to show the feasibility and efficiency of our new method on various signal/image recovery problems, including sparse signal recovery, image inpainting, and image deconvolution. [ABSTRACT FROM AUTHOR]

Thangka, as a precious heritage of painting art, holds irreplaceable research value due to its richness in Tibetan history, religious beliefs, and folk culture. However, it is susceptible to partial damage and form distortion due to natural erosion or inadequate conservation measures. Given the complexity of textures and rich semantics in thangka images, existing image inpainting methods struggle to recover their original artistic style and intricate details. In this paper, we propose a novel approach combining discrete codebook learning with a transformer for image inpainting, tailored specifically for thangka images. In the codebook learning stage, we design an improved network framework based on vector quantization (VQ) codebooks to discretely encode intermediate features of input images, yielding a context-rich discrete codebook. The second phase introduces a parallel transformer module based on a cross-shaped window, which efficiently predicts the index combinations for missing regions under limited computational cost. Furthermore, we devise a multi-scale feature guidance module that progressively fuses features from intact areas with textural features from the codebook, thereby enhancing the preservation of local details in non-damaged regions. We validate the efficacy of our method through qualitative and quantitative experiments on datasets including Celeba-HQ, Places2, and a custom thangka dataset. Experimental results demonstrate that compared to previous methods, our approach successfully reconstructs images with more complete structural information and clearer textural details. [ABSTRACT FROM AUTHOR]

Ancient paintings, as a vital component of cultural heritage, encapsulate a profound depth of cultural significance. Over time, they often suffer from different degradation conditions, leading to damage. Existing ancient painting inpainting methods struggle with semantic discontinuities, blurred textures, and details in missing areas. To address these issues, this paper proposes a generative adversarial network (GAN)-based ancient painting inpainting method named RG-GAN. Firstly, to address the inconsistency between the styles of missing and non-missing areas, this paper proposes a Regional Attention-Style Transfer Module (RASTM) to achieve complex style transfer while maintaining the authenticity of the content. Meanwhile, a multi-scale fusion generator (MFG) is proposed to use the multi-scale residual downsampling module to reduce the size of the feature map and effectively extract and integrate the features of different scales. Secondly, a multi-scale fusion mechanism leverages the Multi-scale Cross-layer Perception Module (MCPM) to enhance feature representation of filled areas to solve the semantic incoherence of the missing region of the image. Finally, the Global Context Perception Discriminator (GCPD) is proposed for the deficiencies in capturing detailed information, which enhances the information interaction across dimensions and improves the discriminator's ability to identify specific spatial areas and extract critical detail information. Experiments on the ancient painting and ancient Huaniao++ datasets demonstrate that our method achieves the highest PSNR values of 34.62 and 23.46 and the lowest LPIPS values of 0.0507 and 0.0938, respectively. [ABSTRACT FROM AUTHOR]

In the domain of handwritten character recognition, inpainting occluded offline characters is essential. Relying on the remarkable achievements of transformers in various tasks, we present a novel framework called “Enhanced Inpainting with Multi-head Attention and stacked long short-term memory (LSTM) Network” (E-Inpaint). This framework aims to restore occluded offline handwriting while capturing its online signal counterpart, enriched with dynamic characteristics. The proposed approach employs Convolutional Neural Network (CNN) and Multi-Layer Perceptron (MLP) in order to extract essential hidden features from the handwriting image. These features are then decoded by stacked LSTM with Multi-head Attention, achieving the inpainting process and generating the online signal corresponding to the uncorrupted version. To validate our work, we utilize the recognition system Beta-GRU on Latin, Indian, and Arabic On/Off dual datasets. The obtained results show the efficiency of using stacked-LSTM network with multi-head attention, enhancing the quality of the restored image and significantly improving the recognition rate using the innovative Beta-GRU system. Our research mainly highlights the potential of E-Inpaint in enhancing handwritten character recognition systems. [ABSTRACT FROM AUTHOR]

Deep learning has brought essential improvement to image inpainting technology. Conventional deep-learning methods primarily focus on creating visually appealing content in the missing parts of images. However, these methods usually generate edge variations and blurry structures in the filled images, which lead to imbalances in quantitative metrics PSNR/SSIM and LPIPS/FID. In this work, we introduce a pioneering model called PTG-Fill, which utilizes a coarse-to-fine architecture to achieve smooth-variation image inpainting. Our approach adopts the novel Stable-Partial Convolution to construct the coarse network, which integrates a smooth mask-update process to ensure its long-term operation. Meanwhile, we propose the novel Distinctive-Gated Convolution to construct the refined network, which diminishes pixel-level variations by the distinctive attention. Additionally, we build up a novel Transformer bridger to preserve the in-depth features for image refinement and facilitate the operation of the two-stage network. Our extensive experiments demonstrate that PTG-Fill outperforms previous state-of-the-art methods both quantitatively and qualitatively under various mask ratios on four benchmark datasets: CelebA-HQ, FFHQ, Paris StreetView, and Places2. Code and pre-trained weights are available at https://github.com/zeyuwang-zju/PTG-Fill. [ABSTRACT FROM AUTHOR]

Current single-view image inpainting methods often suffer from low image information utilization and suboptimal repair outcomes. To address these challenges, this paper introduces a novel image inpainting framework that leverages a structure-guided multi-scale attention pyramid network. This network consists of a structural repair network and a multi-scale attention pyramid semantic repair network. The structural repair component utilizes a dual-branch U-Net network for robust structure prediction under strong constraints. The predicted structural view then serves as auxiliary information for the semantic repair network. This latter network exploits the pyramid structure to extract multi-scale features of the image, which are further refined through an attention feature fusion module. Additionally, a separable gated convolution strategy is employed during feature extraction to minimize the impact of invalid information from missing areas, thereby enhancing the restoration quality. Experiments conducted on standard datasets such as Paris Street View and CelebA demonstrate the superiority of our approach over existing methods through quantitative and qualitative comparisons. Further ablation studies, by incrementally integrating proposed mechanisms into a baseline model, substantiate the effectiveness of our multi-view restoration strategy, separable gated convolution, and multi-scale attention feature fusion. [ABSTRACT FROM AUTHOR]

Removing photobombing elements from images is a challenging task that requires sophisticated image inpainting techniques. Despite the availability of various methods, their effectiveness depends on the complexity of the image and the nature of the distracting element. To address this issue, we conducted a benchmark study to evaluate 10 state-of-the-art photobombing removal methods on a dataset of over 300 images. Our study focused on identifying the most effective image inpainting techniques for removing unwanted regions from images. We annotated the photobombed regions that require removal and evaluated the performance of each method using peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Fréchet inception distance (FID). The results show that image inpainting techniques can effectively remove photobombing elements, but more robust and accurate methods are needed to handle various image complexities. Our benchmarking study provides a valuable resource for researchers and practitioners to select the most suitable method for their specific photobombing removal task. [ABSTRACT FROM AUTHOR]

Deep neural networks with a significant ability to learn and extract image discriminative features make a significant contribution to image retrieval systems. Poor performance in retrieving query images with missing regions is the weak point of image retrieval systems. In this paper, a generative adversarial network is proposed with the aim of inpainting the incomplete images with missing regions in the image retrieval system. Query image inpainting is performed simultaneously at both general and partial levels with two generative networks. Inpainted areas include the semantic and visual features of the input query image. The inpainted image can then be used in the image retrieving system. In the image retrieval process, the content features of the image are extracted from handcrafted features and the VGG-16 deep neural network, including color, texture, and semantic features. The attribute vector of each image is obtained by fusion of the attributes of both parts. Finally, similar images are retrieved based on the smallest Euclidean distance. The explainability of important features of the image in the form of effective super pixels of the image has also been interpreted before and after the use of the LIME technique. The performance of the image retrieval model is confirmed on the ImageNet dataset. [ABSTRACT FROM AUTHOR]

The scalar auxiliary variable (SAV) approach of Shen et al. (2018), which presents a novel way to discretize a large class of gradient flows, has been extended and improved by many authors for general dissipative systems. In this work we consider a Cahn–Hilliard system with mass source that, for image processing and biological applications, may not admit a dissipative structure involving the Ginzburg–Landau energy. Hence, compared to previous works, the stability of SAV-discrete solutions for such systems is not immediate. We establish, with a bounded mass source, stability and convergence of time discrete solutions for a first-order relaxed SAV scheme in the sense of Jiang et al. (2022), and apply our ideas to Cahn–Hilliard systems with mass source appearing in diblock co-polymer phase separation, tumor growth, image inpainting, and segmentation. [ABSTRACT FROM AUTHOR]

Introduction: The knowledge about forest growth, influenced by factors such as tree species, tree age, and environmental conditions, is a key for future forest preservation. Height and age data can be combined to describe forest growth and used to infer known environmental effects. Methods: In this study, we built 14 height growth curves for stands composed of monospecific or mixed species using ground measurements and satellite data. We built a random forest height model from tree species, age, area of disturbance, and 125 environmental parameters (climate, altitude, soil composition, geology, stand ownership, and proximity to road and urban areas). Using feature elimination and SHapley Additive exPlanations (SHAP) analysis, we identified six key features explaining the forest growth and investigated how they affect the height. Results: The agreement between satellite and ground data justifies their simultaneous exploitation. Age and tree species are the main predictors of tree height (49% and 10%, respectively). The disturbed patch area, revealing the regeneration method, impacts post-disturbance growth at 19%. The soil pH, altitude, and climatic water budget in summer impact tree height differently depending on the age and tree species. Discussion: Methods integrating satellite and field data show promise for analyzing future forest evolution. [ABSTRACT FROM AUTHOR]

The synthesis of pseudo-healthy images, involving the generation of healthy counterparts for pathological images, is crucial for data augmentation, clinical disease diagnosis, and understanding pathology-induced changes. Recently, Generative Adversarial Networks (GANs) have shown substantial promise in this domain. However, the heterogeneity of intracranial infection symptoms caused by various infections complicates the model's ability to accurately differentiate between pathological and healthy regions, leading to the loss of critical information in healthy areas and impairing the precise preservation of the subject's identity. Moreover, for images with extensive lesion areas, the pseudo-healthy images generated by these methods often lack distinct organ and tissue structures. To address these challenges, we propose a three-stage method (localization, inpainting, synthesis) that achieves nearly perfect preservation of the subject's identity through precise pseudo-healthy synthesis of the lesion region and its surroundings. The process begins with a Segmentor, which identifies the lesion areas and differentiates them from healthy regions. Subsequently, a Vague-Filler fills the lesion areas to construct a healthy outline, thereby preventing structural loss in cases of extensive lesions. Finally, leveraging this healthy outline, a Generative Adversarial Network integrated with a contextual residual attention module generates a more realistic and clearer image.Our method was validated through extensive experiments across different modalities within the BraTS2021 dataset, achieving a healthiness score of 0.957. The visual quality of the generated images markedly exceeded those produced by competingmethods, with enhanced capabilities in repairing large lesion areas. Further testing on the COVID-19-20 dataset showed that our model could effectively partially reconstruct images of other organs. [ABSTRACT FROM AUTHOR]

Building facade completion is an important part of digitizing the structures of buildings using computer technology. Due to the intricate textures and structures in building facade images, existing image-completion algorithms cannot accurately restore the rich texture and detailed information. In response, this paper proposes a novel network to simultaneously recover the texture and semantic structural features of building facades. By incorporating dynamic convolutions into each layer of the feature encoder, the shallow layers of the completion network can create a global receptive field, thus enhancing the model's feature-extraction capability. Additionally, a spatial attention branch is integrated into the dynamic convolution module to boost the correlation between the completion area and its surrounding edge area, resulting in improved edge clarity and accuracy of the completed facade image. Experimental results on multiple public image datasets demonstrate that the proposed model in this paper achieves state-of-the-art results when applied to real-world datasets. [ABSTRACT FROM AUTHOR]

The article examines the reflection of contemporary artist Muhanned Cader about Sri Lanka. Topics mentioned include a description of Cader's exhibition depicting colonial myths and modes of visuality relating to biology, meteorology, photography and cartography, the history of Ceylon, and a brief highlights of Cader's educational and career background.

This research paper presents a novel approach for occlusion inpainting in thermal images to efficiently segment and enhance obscured regions within these images. The increasing reliance on thermal imaging in fields like surveillance, security, and defense necessitates the accurate detection of obscurants such as smoke and fog. Traditional methods often struggle with these complexities, leading to the need for more advanced solutions. Our proposed methodology uses a Generative Adversarial Network (GAN) to fill occluded areas in thermal images. This process begins with an obscured region segmentation, followed by a GAN-based pixel replacement in these areas. The methodology encompasses building, training, evaluating, and optimizing the model to ensure swift real-time performance. One of the key challenges in thermal imaging is identifying effective strategies to mitigate critical information loss due to atmospheric interference. Our approach addresses this by employing sophisticated deep-learning techniques. These techniques segment, classify and inpaint these obscured regions in a patch-wise manner, allowing for more precise and accurate image restoration. We propose utilizing architectures similar to Pix2Pix and UNet networks for generative and segmentation tasks. These networks are known for their effectiveness in image-to-image translation and segmentation tasks. Our method enhances the segmentation and inpainting process by leveraging their architectural similarities. To validate our approach, we provide a quantitative analysis and performance comparison. We include a quantitative comparison between (Pix2Pix and UNet) and our combined architecture. The comparison focuses on how well each model performs in terms of accuracy and speed, highlighting the advantages of our integrated approach. This research contributes to advancing thermal imaging techniques, offering a more robust solution for dealing with obscured regions. The integration of advanced deep learning models holds the potential to significantly improve image analysis in critical applications like surveillance and security. [ABSTRACT FROM AUTHOR]

In recent years, with the development of 3D digitization of cultural relics, most cultural sites contain a large number of fine 3D data of cultural relics, especially complex geometric objects such as painted cultural relics. At present, how to automatically extract surface damage information from the fine 3D color model of painted cultural relics and avoid the loss of accuracy caused by reducing the dimension using conventional methods is an urgentproblem. In view of the above issues, this paper proposes an automatic and high-precision extraction method for cultural relics surface shedding diseases based on 3D fine data. First, this paper designs a 2D and 3D integrated data conversion model based on OpenSceneGraph, a 3D engine, which performs mutual conversion between 3D color model textures and 2D images. Second, this paper proposes a simple linear iterative clustering segmentation algorithm with an adaptive k value, which solves the problem of setting the superpixel k value and improves the accuracy of image segmentation. Finally, through the 2D and 3D integrated models, the disease is statistically analyzed and labeled on the 3D model. Experiments show that for painted plastic objects with complex surfaces, the disease extraction method based on the 3D fine model proposed in this paper has improved geometric accuracy compared with the current popular orthophoto extraction method, and the disease investigation is more comprehensive. Compared with the current 3D manual extraction method in commercial software, this method greatly improves the efficiency of disease extraction while ensuring extraction accuracy. The research method of this paper activates many existing 3D fine data of cultural protection units and converts conventional 2D data mining and analysis into 3D, which is more in line with the scientific utilization of data in terms of accuracy and efficiency and has certain scientific research value, leading value and practical significance. [ABSTRACT FROM AUTHOR]

Computer vision has advanced much in recent years. Several tasks, such as image recognition, classification, or image restoration, are regularly solved with applications using artificial intelligence techniques. Image restoration comprises different use cases such as style transferring, improvement of quality resolution, or completing missing parts. The latter is also known as image inpainting, virtual image inpainting in this case, which consists of reconstructing missing regions or elements. This paper explores how to evaluate the performance of a deep learning method to do virtual image inpainting to reconstruct missing architectonical elements in images of ruined Greek temples to measure the performance of different activation functions. Unlike a previous study related to this work, a direct reconstruction process without segmented images was used. Then, two evaluation methods are presented: the objective one (mathematical metrics) and an expert (visual perception) evaluation to measure the performance of the different approaches. Results conclude that ReLU outperforms other activation functions, while Mish and Leaky ReLU perform poorly, and Swish's professional evaluations highlight a gap between mathematical metrics and human visual perception. [ABSTRACT FROM AUTHOR]

A deep‐learning algorithm is proposed for the inpainting of Bragg coherent diffraction imaging (BCDI) patterns affected by detector gaps. These regions of missing intensity can compromise the accuracy of reconstruction algorithms, inducing artefacts in the final result. It is thus desirable to restore the intensity in these regions in order to ensure more reliable reconstructions. The key aspect of the method lies in the choice of training the neural network with cropped sections of diffraction data and subsequently patching the predictions generated by the model along the gap, thus completing the full diffraction peak. This approach enables access to a greater amount of experimental data for training and offers the ability to average overlapping sections during patching. As a result, it produces robust and dependable predictions for experimental data arrays of any size. It is shown that the method is able to remove gap‐induced artefacts on the reconstructed objects for both simulated and experimental data, which becomes essential in the case of high‐resolution BCDI experiments. [ABSTRACT FROM AUTHOR]

Flow field data obtained by particle image velocimetry (PIV) could include isolated large damaged areas that are caused by the refractive index, light transmittance, and tracking capability of particles. The traditional deep learning reconstruction methods of PIV fluid data are all based on the velocity field database, and these methods could not achieve satisfactory results for large flow field missing areas. We propose a new reconstruction method of fluid data using PIV particle images. Since PIV particle images are the source of PIV velocity field data, particle images include more complete underlying information than velocity field data. We study the application of PIV experimental particle database in the reconstruction of flow field data using deep generative networks (GAN). To verify the inpainting effect of velocity field using PIV particle images, we design two semantic inpainting methods based on two GAN models with PIV particle image database and PIV fluid velocity database, respectively. Then, the qualitative and quantitative inpainting results of two PIV databases are compared on different metrics. For the reconstruction of velocity field, the mean relative error of using the particle image database could achieve a 52% reduction compared to a velocity database. For the reconstruction of vorticity field, the maximal and mean relative errors can reduce by 50% when using the particle image database. The maximum inpainting errors of two database inputs are both mainly concentrated on the turbulence vortex area, which means the reconstruction of complex non-Gaussian distribution of turbulence vortex is a problem for semantic inpainting of the experimental data. [ABSTRACT FROM AUTHOR]

The rapid advancement of deep learning technologies presents novel opportunities for restoring damaged patterns in ancient silk, which is pivotal for the preservation and propagation of ancient silk culture. This study systematically scrutinizes the evolutionary trajectory of image inpainting algorithms, with a particular emphasis on those firmly rooted in the Context-Encoder structure. To achieve this study's objectives, a meticulously curated dataset comprising 6996 samples of ancient Chinese silk (256 × 256 pixels) was employed. Context-Encoder-based image inpainting models—LISK, MADF, and MEDFE—were employed to inpaint damaged patterns. The ensuing restoration effects underwent rigorous evaluation, providing a comprehensive analysis of the inherent strengths and limitations of each model. This study not only provides a theoretical foundation for adopting image restoration algorithms grounded in the Context-Encoder structure but also offers ample scope for exploration in achieving more effective restorations of ancient damaged silk. [ABSTRACT FROM AUTHOR]

Automatic micro-defect detection is crucial for promoting efficiency in the production lines of patterned OLED panels. Recently, deep learning algorithms have emerged as promising solutions for micro-defect detection. However, in real-world industrial scenarios, the scarcity of training data or annotations results in a drop in performance. A multi-stage few-shot micro-defect detection approach is proposed for patterned OLED panels to deal with this problem. Firstly, we introduce a converter from defective to defect-free images based on our redesigned Vector Quantized-Variational AutoEncoder (VQ-VAE), aiming to inpaint defects with normal textures. Next, we exploit a region-growing method with automatic seed points to obtain the defect's segmentation and geometric parameters in each image block. Reliable seed points are provided by structural similarity index maps between defective sub-blocks and reconstructed reference. Finally, a multi-scale Siamese neural network is proposed to identify the category of extracted defects. With our proposed approach, detection and classification results of defects can be obtained successively. Our experimental results on samples at different array processes demonstrate the superb adaptability of VQ-VAE, with a defect detection rate ranging from 90.0% to 96.0%. Additionally, compared with existing classification models, our multi-scale Siamese neural network exhibits an impressive 98.6% classification accuracy for a long-tailed defect dataset without overfitting. In summary, the proposed approach shows great potential for practical micro-defect detection in industrial scenarios with limited training data. [ABSTRACT FROM AUTHOR]