Search

In recent years generative AI models and tools have experienced a significant increase, especially techniques to generate synthetic multimedia content, such as images or videos. These methodologies present a wide range of possibilities; however, they can also present several risks that should be taken into account. In this survey we describe in detail different techniques for generating synthetic multimedia content, and we also analyse the most recent techniques for their detection. In order to achieve these objectives, a key aspect is the availability of datasets, so we have also described the main datasets available in the state of the art. Finally, from our analysis we have extracted the main trends for the future, such as transparency and interpretability, the generation of multimodal multimedia content, the robustness of models and the increased use of diffusion models. We find a roadmap of deep challenges, including temporal consistency, computation requirements, generalizability, ethical aspects, and constant adaptation. [ABSTRACT FROM AUTHOR]

Recent advancements in low-rank adaptation (LoRA) have shown its effectiveness in fine-tuning diffusion models for generating images tailored to new downstream tasks. Research on integrating multiple LoRA modules to accommodate new tasks has also gained traction. One emerging approach constructs several LoRA modules, but more than three typically decrease the generation performance of pre-trained models. The mixture-of-experts model solves the performance issue, but LoRA modules are not combined using text prompts; hence, generating images by combining LoRA modules does not dynamically reflect the user's desired requirements. This paper proposes a LoRA fusion method that applies an attention mechanism to effectively capture the user's text-prompting intent. This method computes the cosine similarity between predefined keys and queries and uses the weighted sum of the corresponding values to generate task-specific LoRA modules without the need for retraining. This method ensures stability when merging multiple LoRA modules and performs comparably to fully retrained LoRA models. The technique offers a more efficient and scalable solution for domain adaptation in large language models, effectively maintaining stability and performance as it adapts to new tasks. In the experiments, the proposed method outperformed existing methods in text–image alignment and image similarity. Specifically, the proposed method achieved a text–image alignment score of 0.744, surpassing an SVDiff score of 0.724, and a normalized linear arithmetic composition score of 0.698. Moreover, the proposed method generates superior semantically accurate and visually coherent images. [ABSTRACT FROM AUTHOR]

Automatic signature verification has been widely studied for authentication purposes in real life, but limited data availability still poses a significant challenge. To address this issue, we propose a method with a denoising diffusion probabilistic model (DDPM) to generate artificial signatures that closely resemble authentic ones. In the proposed method, we modified the noise prediction process of the DDPM to allow the generation of signatures specific to certain classes. We also employed an auxiliary classification process to ensure that the generated signatures closely resemble the originals. The model was trained and evaluated using the CEDAR signature dataset, a widely used collection of offline handwritten signatures for signature verification research. The results indicate that the generated signatures exhibited a high similarity to the originals, with an average structural similarity index (SSIM) of 0.9806 and a root mean square error (RMSE) of 0.1819. Furthermore, when the generated signatures were added to the training data and the signature verification model was retrained and validated, the model achieved an accuracy of 94.87% on the test data, representing an improvement of 0.061 percentage points compared to training on only the original dataset. These results indicate that the generated signatures reflect the diversity that original signatures may exhibit and that the generated data can enhance the performance of verification systems. The proposed method introduces a novel approach to utilizing DDPM for signature data generation and demonstrates that the auxiliary classification process can reduce the likelihood of generated data being mistaken for forged signatures. [ABSTRACT FROM AUTHOR]

This study aims to explore an autoencoder-based method for generating brain MRI images of patients with Autism Spectrum Disorder (ASD) and non-ASD individuals, and to discriminate ASD based on the generated images. Initially, we introduce the research background of ASD and related work, as well as the application of deep learning in the field of medical imaging. Subsequently, we detail the architecture and training process of the proposed autoencoder model, and present the results of generating MRI images for ASD and non-ASD patients. Following this, we designed an ASD classifier based on the generated images and elucidated its structure and training methods. Finally, through analysis and discussion of experimental results, we validated the effectiveness of the proposed method and explored future research directions and potential clinical applications. This research offers new insights and methodologies for addressing challenges in ASD studies using deep learning technology, potentially contributing to the automated diagnosis and research of ASD. [ABSTRACT FROM AUTHOR]

Purpose: Total-body dynamic positron emission tomography (PET) imaging with total-body coverage and ultrahigh sensitivity has played an important role in accurate tracer kinetic analyses in physiology, biochemistry, and pharmacology. However, dynamic PET scans typically entail prolonged durations ( 60 minutes), potentially causing patient discomfort and resulting in artifacts in the final images. Therefore, we propose a dynamic frame prediction method for total-body PET imaging via deep learning technology to reduce the required scanning time. Methods: On the basis of total-body dynamic PET data acquired from 13 subjects who received [68Ga]Ga-FAPI-04 (68Ga-FAPI) and 24 subjects who received [68Ga]Ga-PSMA-11 (68Ga-PSMA), we propose a bidirectional dynamic frame prediction network that uses the initial and final 10 min of PET imaging data (frames 1–6 and frames 25–30, respectively) as inputs. The peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) were employed as evaluation metrics for an image quality assessment. Moreover, we calculated parametric images (68Ga-FAPI: , 68Ga-PSMA: ) based on the supplemented sequence data to observe the quantitative accuracy of our approach. Regions of interest (ROIs) and statistical analyses were utilized to evaluate the performance of the model. Results: Both the visual and quantitative results illustrate the effectiveness of our approach. The generated dynamic PET images yielded PSNRs of 36.056 ± 0.709 dB for the 68Ga-PSMA group and 33.779 ± 0.760 dB for the 68Ga-FAPI group. Additionally, the SSIM reached 0.935 ± 0.006 for the 68Ga-FAPI group and 0.922 ± 0.009 for the 68Ga-PSMA group. By conducting a quantitative analysis on the parametric images, we obtained PSNRs of 36.155 ± 4.813 dB (68Ga-PSMA, ) and 43.150 ± 4.102 dB (68Ga-FAPI, ). The obtained SSIM values were 0.932 ± 0.041 (68Ga-PSMA) and 0.980 ± 0.011 (68Ga-FAPI). The ROI analysis conducted on our generated dynamic PET sequences also revealed that our method can accurately predict temporal voxel intensity changes, maintaining overall visual consistency with the ground truth. Conclusion: In this work, we propose a bidirectional dynamic frame prediction network for total-body 68Ga-PSMA and 68Ga-FAPI PET imaging with a reduced scan duration. Visual and quantitative analyses demonstrated that our approach performed well when it was used to predict one-hour dynamic PET images. https://github.com/OPMZZZ/BDF-NET. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) have achieved remarkable progress in the natural image field. However, when applying GANs in the remote sensing (RS) image generation task, an extraordinary phenomenon is observed: the GAN model is more sensitive to the amount of training data for RS image generation than for natural image generation (Fig. 1). In other words, the generation quality of RS images will change significantly with the number of training categories or samples per category. In this paper, we first analyze this phenomenon from two kinds of toy experiments and conclude that the amount of feature information contained in the GAN model decreases with reduced training data (Fig. 2). Then we establish a structural causal model (SCM) of the data generation process and interpret the generated data as the counterfactuals. Based on this SCM, we theoretically prove that the quality of generated images is positively correlated with the amount of feature information. This provides insights for enriching the feature information learned by the GAN model during training. Consequently, we propose two innovative adjustment schemes, namely uniformity regularization and entropy regularization, to increase the information learned by the GAN model at the distributional and sample levels, respectively. Extensive experiments on eight RS datasets and three natural datasets show the effectiveness and versatility of our methods. The source code is available at https://github.com/rootSue/Causal-RSGAN. [ABSTRACT FROM AUTHOR]

This paper introduces an innovative framework for visualisation design ideation, which includes a collection of terms for creative visualisation design, the five-step process, and an implementation called VisAlchemy. Throughout the visualisation ideation process, individuals engage in exploring various concepts, brainstorming, sketching ideas, prototyping, and experimenting with different methods to visually represent data or information. Sometimes, designers feel incapable of sketching, and the ideation process can be quite lengthy. In such cases, generative AI can provide assistance. However, even with AI, it can be difficult to know which vocabulary to use and how to strategically approach the design process. Our strategy prompts imaginative and structured narratives for generative AI use, facilitating the generation and refinement of visualisation design ideas. We aim to inspire fresh and innovative ideas, encouraging creativity and exploring unconventional concepts. VisAlchemy is a five-step framework: a methodical approach to defining, exploring, and refining prompts to enhance the generative AI process. The framework blends design elements and aesthetics with context and application. In addition, we present a vocabulary set of 300 words, underpinned from a corpus of visualisation design and art papers, along with a demonstration tool called VisAlchemy. The interactive interface of the VisAlchemy tool allows users to adhere to the framework and generate innovative visualisation design concepts. It is built using the SDXL Turbo language model. Finally, we demonstrate its use through case studies and examples and show the transformative power of the framework to create inspired and exciting design ideas through refinement, re-ordering, weighting of words and word rephrasing. [ABSTRACT FROM AUTHOR]

Cytology, a type of pathological examination, involves sampling cells from the human body and observing the morphology of the nucleus, cytoplasm, and cell arrangement. In developing classification AI technologies to support cytology, it is essential to collect and utilize a diverse range of images without bias. However, this is often challenging in practice because of the epidemiologic bias of cancer types and cellular characteristics. The main aim of this study was to develop a method to generate cytological diagnostic images from image findings using text-to-image technology in order to generate diverse images. In the proposed method, we collected Papanicolaou-stained specimens derived from the lung cells of 135 lung cancer patients, from which we extracted 472 patch images. Descriptions of the corresponding findings for these patch images were compiled to create a data set. This dataset was then utilized to finetune the Stable Diffusion (SD) v1 and v2 models. The cell images generated by this method closely resemble real images, and both cytotechnologists and cytopathologists provided positive subjective evaluations. Furthermore, SDv2 demonstrated shapes and contours of nuclei and cytoplasm that were more similar to real images compared to SDv1, showing superior performance in quantitative evaluation metrics. When the generated images were utilized in the classification tasks for cytological images, there was an improvement in classification performance. These results indicate that the proposed method may be effective for generating high-quality cytological images, which enables the image classification model to learn diverse features, thereby improving classification performance. [ABSTRACT FROM AUTHOR]

Traditional multimedia experiences are undergoing a transformation as generative AI integration fosters enhanced creative workflows, streamlines content creation processes, and unlocks the potential for entirely new forms of multimedia storytelling. It has potential to generate captivating visuals to accompany a documentary based solely on historical text descriptions, or creating personalized and interactive multimedia experiences tailored to individual user preferences. From the high-resolution cameras in our smartphones to the immersive experiences offered by the latest technologies, the impact of generative imaging undeniable. This study delves into the burgeoning field of generative AI, with a focus on its revolutionary impact on image generation. It explores the background of traditional imaging in consumer electronics and the motivations for integrating AI, leading to enhanced capabilities in various applications. The research critically examines current advancements in state-of-the-art technologies like DALL-E 2, Craiyon, Stable Diffusion, Imagen, Jasper, NightCafe, and Deep AI, assessing their performance on parameters such as image quality, diversity, and efficiency. It also addresses the limitations and ethical challenges posed by this integration, balancing creative autonomy with AI automation. The novelty of this work lies in its comprehensive analysis and comparison of these AI systems, providing insightful results that highlight both their strengths and areas for improvement. The conclusion underscores the transformative potential of generative AI in image generation, paving the way for future research and development to further enhance and refine these technologies. This article serves as a critical guide for understanding the current landscape and future prospects of AI-driven image creation, offering a glimpse into the evolving synergy between human creativity and artificial intelligence. [ABSTRACT FROM AUTHOR]

This study aims to explore an autoencoder-based method for generating brain MRI images of patients with Autism Spectrum Disorder (ASD) and non-ASD individuals, and to discriminate ASD based on the generated images. Initially, we introduce the research background of ASD and related work, as well as the application of deep learning in the field of medical imaging. Subsequently, we detail the architecture and training process of the proposed autoencoder model, and present the results of generating MRI images for ASD and non-ASD patients. Following this, we designed an ASD classifier based on the generated images and elucidated its structure and training methods. Finally, through analysis and discussion of experimental results, we validated the effectiveness of the proposed method and explored future research directions and potential clinical applications. This research offers new insights and methodologies for addressing challenges in ASD studies using deep learning technology, potentially contributing to the automated diagnosis and research of ASD. [ABSTRACT FROM AUTHOR]

Archaeological illustration is a graphic recording technique that delineates the shape, structure, and ornamentation of cultural artifacts using lines, serving as vital material in archaeological work and scholarly research. Aiming at the problems of low line accuracy in the results of current mainstream image generation algorithms and interference caused by severe mural damage, this paper proposes a mural archaeological illustration generation algorithm based on multi-branch feature cross fusion (U2FGAN). The algorithm optimizes skip connections in U2Net through a channel attention mechanism, constructing a multi-branch generator consisting of a line extractor and an edge detector, which separately identify line features and edge information in artifact images before fusing them to generate accurate, high-resolution illustrations. Additionally, a multi-scale conditional discriminator is incorporated to guide the generator in outputting high-quality illustrations with clear details and intact structures. Experiments conducted on the Dunhuang mural illustration datasets demonstrate that compared to mainstream counterparts, U2FGAN reduced the Mean Absolute Error (MAE) by 10.8% to 26.2%, while also showing substantial improvements in Precision (by 9.8% to 32.3%), Fβ-Score (by 5.1% to 32%), and PSNR (by 0.4 to 2.2 dB). The experimental results show that the proposed method outperforms other mainstream algorithms in archaeological illustration generation. [ABSTRACT FROM AUTHOR]

In recent years, Generative Adversarial Network (GAN) have shown great potential and achieved excellent performance on the task of generating optical images. However, when GAN is applied to Synthetic Aperture Radar (SAR) images, the differences in imaging mechanisms between SAR images and optical images make GAN susceptible to training instability and model collapse problems during training. In this paper, we propose a new end-to-end model called Dual Discriminator Modified Conditional Generative Adversarial Network (DDM-CGAN) to address these issues. First, two discriminators are designed to play an adversarial game against the generator in DDM-CGAN. One discriminator favors samples from real data, while the other rewards high scores for generated samples. Essentially, we designed a novel objective function by utilizing the dual discriminator to combine the respective advantages of alternative cost function and original cost function of the standard GAN. We theoretically prove that this objective function can optimize DDM-CGAN towards minimizing the Kullback–Leibler Divergence, thus avoiding the problem of non-convergence during model training. Second, we propose a modified gradient penalty, which makes the model training more stable. In addition, the introduction of a discriminative auxiliary classifier provides the generator with information about the target distribution, thus improving the diversity of the generated images. We perform comprehensive qualitative and quantitative experiments with the Gaofen-3 SAR image dataset. Our proposed DDM-CGAN is compared with the state-of-the-art SAR image generation methods. Experimental results demonstrate that the SAR images generated by DDM-CGAN achieves optimal results in terms of the similarity, statistical characteristics, and diversity. [ABSTRACT FROM AUTHOR]

• Language-based Subject Generation faces challenge in accurate portrayal of subject. • Nowadays Reference Guided Generation lacks ability to preserve subject identity. • Exemplar-based instructions with visual tokens preserve visual details of subject. • Model based guidance samples better quality images with different pose. • Our model achieved highest CLIP, DINO score and user study compared to others. This paper presents a novel approach to subject-driven image generation that addresses the limitations of traditional text-to-image diffusion models. Our method generates images using reference images without relying on language-based prompts. We introduce a visual detail preserving module that captures intricate details and textures, addressing overfitting issues associated with limited training samples. The model's performance is further enhanced through a modified classifier-free guidance technique and feature concatenation, enabling the natural positioning and harmonization of subjects within diverse scenes. Quantitative assessments using CLIP, DINO and Quality scores (QS), along with a user study, demonstrate the superior quality of our generated images. Our work highlights the potential of pre-trained models and visual patch embeddings in subject-driven editing, balancing diversity and fidelity in image generation tasks. Our implementation is available at https://github.com/8eomio/Subject-Inpainting. [Display omitted] To create your abstract, type over the instructions in the template box below. Fonts or abstract dimensions should not be changed or altered. [ABSTRACT FROM AUTHOR]

The advent of deep neural networks and improved computational power have brought a revolutionary transformation in the fields of computer vision and image processing. Within the realm of computer vision, there has been a significant interest in the area of synthetic image generation, which is a creative side of AI. Many researchers have introduced innovative methods to identify deep neural network‐based architectures involved in image generation via different modes of input, like text, scene graph layouts and so forth to generate synthetic images. Computer‐generated images have been found to contribute a lot to the training of different machine and deep‐learning models. Nonetheless, we have observed an immediate need for a comprehensive and systematic literature review that encompasses a summary and critical evaluation of current primary studies' approaches toward image generation. To address this, we carried out a systematic literature review on synthetic image generation approaches published from 2018 to February 2023. Moreover, we have conducted a systematic review of various datasets, approaches to image generation, performance metrics for existing methods, and a brief experimental comparison of DCGAN (deep convolutional generative adversarial network) and cGAN (conditional generative adversarial network) in the context of image generation. Additionally, we have identified applications related to image generation models with critical evaluation of the primary studies on the subject matter. Finally, we present some future research directions to further contribute to the field of image generation using deep neural networks. [ABSTRACT FROM AUTHOR]

Recent work has shown that Neural Ordinary Differential Equations (ODEs) can serve as generative models of images using the perspective of Continuous Normalizing Flows (CNFs). Such models offer exact likelihood calculation, and invertible generation/density estimation. In this work we introduce a Multi-Resolution variant of such models (MRCNF), by characterizing the conditional distribution over the additional information required to generate a fine image that is consistent with the coarse image. We introduce a transformation between resolutions that allows for no change in the log likelihood. We show that this approach yields comparable likelihood values for various image datasets, with improved performance at higher resolutions, with fewer parameters, using only one GPU. Further, we examine the out-of-distribution properties of MRCNFs, and find that they are similar to those of other likelihood-based generative models. [ABSTRACT FROM AUTHOR]

In this paper we present a novel multi-attribute face manipulation method based on textual descriptions. Previous text-based image editing methods either require test-time optimization for each individual image or are restricted to single attribute editing. Extending these methods to multi-attribute face image editing scenarios will introduce undesired excessive attribute change, e.g., text-relevant attributes are overly manipulated and text-irrelevant attributes are also changed. In order to address these challenges and achieve natural editing over multiple face attributes, we propose a new decoupling training scheme where we use group sampling to get text segments from same attribute categories, instead of whole complex sentences. Further, to preserve other existing face attributes, we encourage the model to edit the latent code of each attribute separately via an entropy constraint. During the inference phase, our model is able to edit new face images without any test-time optimization, even from complex textual prompts. We show extensive experiments and analysis to demonstrate the efficacy of our method, which generates natural manipulated faces with minimal text-irrelevant attribute editing. Code and pre-trained model are available at https://github.com/hwang1996/ManiCLIP. [ABSTRACT FROM AUTHOR]

Copyright of Journal of South China University of Technology (Natural Science Edition) is the property of South China University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Although this research has made significant progress in image generation models, they still face issues such as insufficient diversity of generated images, poor quality of high-resolution images, and the need for a large amount of training data for model optimization. This paper studies poster design based on deep learning automatic image generation algorithm, using a recursive supervised image generation algorithm framework of generative adversarial networks for multi-view image generation and super-resolution generation tasks of small sample digital poster images. Various improvements have been proposed to enhance the performance of the GAN network model for poster design image generation tasks. Based on experimental research, this paper's model uses generative adversarial networks to distinguish randomly cropped low resolution and high-resolution poster blocks, ensuring that high-resolution posters maintain their original resolution canvas texture and brush strokes, effectively improving the automatic generation effect of poster images. The evaluation results show that the quantitative evaluation of the proposed algorithm model in knowledge management is distributed in a reasonable range, which indicates that the proposed algorithm model has good performance in knowledge management. The poster design model based on deep learning automatic image generation algorithm proposed in this paper has certain effects. In subsequent practice, the automatic image generation algorithm can be combined with practical needs to improve the efficiency and design effect of poster design. [ABSTRACT FROM AUTHOR]

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)