Search

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub., Funding Agencies|ITEA/VINNOVA project ASSIST [2021-01420]; LiU Cancer; VINNOVA AIDA [M22-0088]; Ake Wiberg foundation; Wallenberg Center for Molecular Medicine as an associated clinical fellow; [2021-01954]

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

Voice conversion systems have become increasingly important as the use of voice technology grows. Deep learning techniques, specifically generative adversarial networks (GANs), have enabled significant progress in the creation of synthetic media, including the field of speech synthesis. One of the most recent examples, StarGAN-VC, uses a single pair of generator and discriminator to convert voices between multiple speakers. However, the training stability of GANs can be an issue. The Top-K methodology, which trains the generator using only the best K generated samples that “fool” the discriminator, has been applied to image tasks and simple GAN architectures. In this work, we demonstrate that the Top-K methodology can improve the quality and stability of converted voices in a state-of-the-art voice conversion system like StarGAN-VC. We also explore the optimal time to implement the Top-K methodology and how to reduce the value of K during training. Through both quantitative and qualitative studies, it was found that the Top-K methodology leads to quicker convergence and better conversion quality compared to regular or vanilla training. In addition, human listeners perceived the samples generated using Top-K as more natural and were more likely to believe that they were produced by a human speaker. The results of this study demonstrate that the Top-K methodology can effectively improve the performance of deep learning-based voice conversion systems., QC 20240110

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub., Funding Agencies|ITEA/VINNOVA project ASSIST [2021-01420]; LiU Cancer; VINNOVA AIDA [M22-0088]; Ake Wiberg foundation; Wallenberg Center for Molecular Medicine as an associated clinical fellow; [2021-01954]

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub., Funding Agencies|ITEA/VINNOVA project ASSIST [2021-01420]; LiU Cancer; VINNOVA AIDA [M22-0088]; Ake Wiberg foundation; Wallenberg Center for Molecular Medicine as an associated clinical fellow; [2021-01954]

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

4D MRI with high spatiotemporal resolution is desired for image-guided liver radiotherapy. Acquiring densely sampled k-space data is time-consuming. Accelerated acquisition with sparse sampling is desirable but often causes degraded image quality or long reconstruction time. We propose the Reconstruct Paired Conditional Generative Adversarial Network (Re-Con-GAN) for shortening the 4D MRI reconstruction time while maintaining the reconstruction quality. Methods: Patients underwent free-breathing liver 4D MRI were included in the study. Fully- and retrospectively under-sampled data at 3, 6 and 10 times (3x, 6x and 10x) were first reconstructed using the nuFFT algorithm. Re-Con-GAN then trained input and output in pairs. Three types of networks, ResNet9, UNet and reconstruction swin transformer, were explored as generators. PatchGAN was selected as the discriminator. Re-Con-GAN processed the data (3D+t) as temporal slices (2D+t). A total of 48 patients with 12332 temporal slices were split into training (37 patients with 10721 slices) and test (11 patients with 1611 slices). Compressed sensing (CS) reconstruction with spatiotemporal sparsity constraint was used as a benchmark. Reconstructed image quality was further evaluated with a liver gross tumor volume (GTV) localization task using Mask-RCNN trained from a separate 3D static liver MRI dataset (70 patients; 103 GTV contours). Results: Re-Con-GAN consistently achieved comparable/better PSNR, SSIM, and RMSE scores compared to CS/UNet models. The inference time of Re-Con-GAN, UNet and CS are 0.15s, 0.16s, and 120s. The GTV detection task showed that Re-Con-GAN and CS, compared to UNet, better improved the dice score (3x Re-Con-GAN 80.98%; 3x CS 80.74%; 3x UNet 79.88%) of unprocessed under-sampled images (3x 69.61%). Conclusion: A generative network with adversarial training is proposed with promising and efficient reconstruction results demonstrated on an in-house dataset. The rapid and qualitative reconstruction of

The demand for specialized hardware to train AI models has increased in tandem with the increase in the model complexity over the recent years. Graphics processing unit (GPU) is one such hardware that is capable of parallelizing operations performed on a large chunk of data. Companies like Nvidia, AMD, and Google have been constantly scaling-up the hardware performance as fast as they can. Nevertheless, there is still a gap between the required processing power and processing capacity of the hardware. To increase the hardware utilization, the software has to be optimized too. In this paper, we present some general GPU optimization techniques we used to efficiently train the optiGAN model, a Generative Adversarial Network that is capable of generating multidimensional probability distributions of optical photons at the photodetector face in radiation detectors, on an 8GB Nvidia Quadro RTX 4000 GPU. We analyze and compare the performances of all the optimizations based on the execution time and the memory consumed using the Nvidia Nsight Systems profiler tool. The optimizations gave approximately a 4.5x increase in the runtime performance when compared to a naive training on the GPU, without compromising the model performance. Finally we discuss optiGANs future work and how we are planning to scale the model on GPUs.

Background and objectives: This study aims to develop and evaluate NeuNN, a system based on convolutional neural networks (CNN) and generative adversarial networks (GAN) for the automatic identification of normal neutrophils and those containing several types of inclusions or showing hypogranulation. Methods: From peripheral blood smears, a set of 5605 digital images was obtained with neutrophils belonging to seven categories: Normal neutrophils (NEU), Hypogranulated (HYP) or containing cryoglobulins (CRY), Döhle bodies (DB), Howell-Jolly body-like inclusions (HJBLI), Green-blue inclusions of death (GBI) and phagocytosed bacteria (BAC). The dataset utilized in this study has been made publicly available. The class of GBI was augmented using synthetic images generated by GAN. The NeuNN classification model is based on an EfficientNet-B7 architecture trained from scratch. Results: NeuNN achieved an overall performance of 94.3% accuracy on the test data set. Performance metrics, including sensitivity, specificity, precision, F1-Score, Jaccard index, and Matthews correlation coefficient indicated overall values of 94%, 99.1%, 94.3%, 94.3%, 89.6%, and 93.6%, respectively. Conclusions: The proposed approach, combining data augmentation and classification techniques, allows for automated identification of morphological findings in neutrophils, such us inclusions or hypogranulation. The system can be used as a support tool for clinical pathologists to detect these specific abnormalities with clinical relevance., Peer Reviewed, Postprint (published version)

Este proyecto implica la creación de una librería en Pytorch que incluye varios métodos para generar datos tabulares artificiales, como GANs, VAEs, Bit Diffusion y un modelo llamado Data Augmentation basado en autoencoders de eliminación de ruido. El objetivo de la librería es proporcionar herramientas accesibles para que los usuarios experimenten y evalúen enfoques generativos. La evaluación de los modelos muestra que su rendimiento varía según el contexto y las características del conjunto de datos. Aunque los resultados son buenos, aún hay margen para incorporar metodologías adicionales y mejorar la calidad de los datos sintéticos. En resumen, la librería ha logrado generar datos artificiales de buena calidad, y con más dedicación, se puede mejorar aún más. Esto establece una base sólida para futuras investigaciones en generación de datos sintéticos., This project involves the creation of a Pytorch library that includes various methods for generating artificial tabular data, such as GANs, VAEs, Bit Diffusion, and a model called Data Augmentation based on denoising autoencoders. The library's purpose is to provide accessible tools for users to experiment with and evaluate generative approaches. The evaluation of the models reveals that their performance varies depending on the context and dataset characteristics. Although the results are good, there is still room to incorporate additional methodologies and improve the quality of synthetic data. In summary, the library has successfully generated high-quality artificial data, and with further dedication, further improvements can be made. This establishes a solid foundation for future research in synthetic data generation.

Tesi en modalitat de compendi de publicacions, (English) Clinical pathologists focus on detailed morphological analysis of peripheral blood (PB) cells for the recognition of abnormal cells and contribute to the diagnosis of hematological diseases. While this traditional method is often precise, it requires significant expertise, is time-consuming, and is subject to variability, especially when discerning subtle cell characteristics. In light of these challenges, this dissertation proposes the innovative use of deep learning, specifically convolutional neural networks (CNNs) and generative adversarial networks (GANs), to add objectivity in morphological examination of PB cells. The research is developed through a three-fold strategy. First, a novel GAN-based model standardizes PB cell image visualization across varied staining techniques. Second, optimized CNN architectures are tailored for precise classification of the selected PB cell types, with emphasis on reactive and malignant blood cells. Results indicate that the introduced Stain Normalization Model effectively manages staining variations while preserving cell morphology. Furthermore, CNN models, based on these normalized data, contribute significantly to the automatic recognition of malignant cells. And third, a modular GAN system generates synthetic PB images to enrich model training, particularly of low-prevalence cell data with special diagnostic interest. This research emphasizes the transformative impact of combining the expertise of clinical pathologists with advanced computational strategies. While these systems serve as potent support tools, it is imperative to note that the clinical pathologist retains the ultimate authority in decision-making. The fusion of traditional expertise and modern technology is a promising trend in the present and near future, enhancing both objectivity and efficiency. This study signifies an advance towards the establishment of an integrated system for an automated screening tool for hematological diseases., (Català) Els patòlegs clínics depenen de l'anàlisi morfològica detallada de les cèl·lules de sang perifèrica (PB) per a un diagnòstic precís de malalties hematològiques. Encara que aquest mètode tradicional és sovint precís, requereix una gran experiència, és laboriós i està subjecte a variabilitat, especialment quan es tracta de discernir característiques subtils de les cèl·lules. Davant d'aquests desafiaments, aquesta tesi doctoral introdueix l'ús innovador de l'aprenentatge profund, específicament xarxes neuronals convolucionals (CNN) i xarxes generatives antagonistes (GAN), per transformar l'examen morfològic de les cèl·lules de la sang. La recerca es desenvolupa a través d'una estratègia de tres vessants: Primer, un model nou basat en GAN estandarditza la visualització de les imatges a través de diverses tècniques de tinció. En segon lloc, s'adapten arquitectures de CNN optimitzades per a una classificació precisa dels tipus principals de cèl·lules sanguínies, amb èmfasi en els limfòcits reactius i les cèl·lules malignes. En tercer lloc, un sistema GAN modular genera imatges sintètiques de les cèl·lules, especialment en casos de cèl·lules de baixa prevalença per enriquir l'entrenament del model. Els resultats indiquen que el Model de Normalització de Tinció introduït gestiona eficaçment les variacions de tinció sense afectar la morfologia cel·lular. A més, els models CNN, construïts amb aquestes dades normalitzades, excel·leixen en la detecció de cèl·lules malignes. El model SyntheticCellGAN enriqueix encara més el conjunt de dades d'entrenament amb cèl·lules sintètiques diagnòsticament rellevants. Aquesta recerca destaca l'impacte transformador de combinar l'experiència dels patòlegs clínics amb eines computacionals avançades. Tot i que aquests sistemes serveixen com a eines de suport, és imperatiu tenir en compte que el patòleg clínic reté la màxima autoritat en la presa de decisions. La fusió de l'experiència tradicional i la tecnologia moderna promet revolu, (Español) Los patólogos clínicos dependen del análisis morfológico detallado de las células de sangre periférica (PB) para un diagnóstico preciso de enfermedades hematológicas. Aunque este método tradicional es a menudo preciso, requiere una gran experiencia, es laborioso y está sujeto a variabilidad, especialmente cuando se trata de discernir características sutiles de las células. Ante estos desafíos, esta tesis doctoral introduce el uso innovador del aprendizaje profundo, específicamente redes neuronales convolucionales (CNN) y redes generativas antagónicas (GAN), para transformar el examen morfológico de las células PB. La investigación se desarrolla a través de una estrategia triple. Primero, un modelo novedoso basado en GAN estandariza la visualización de imágenes de células PB a través de diversas técnicas de tinción. En segundo lugar, se adaptan arquitecturas de CNN optimizadas para una clasificación precisa de los tipos principales de células sanguíneas, con énfasis en los linfocitos reactivos y las células malignas. En tercer lugar, un sistema GAN modular genera imágenes sintéticas de PB, especialmente en el caso de células de baja prevalencia, para enriquecer el entrenamiento del modelo. Los resultados indican que el Modelo de Normalización de Tinción introducido gestiona eficazmente las variaciones de tinción sin afectar la morfología celular. Además, los modelos CNN, construidos con estos datos normalizados, sobresalen en la detección de células malignas. El modelo SyntheticCellGAN enriquece aún más el conjunto de datos de entrenamiento con células sintéticas con relevancia diagnóstica. Esta investigación destaca el impacto transformador de combinar la experiencia de los patólogos clínicos con herramientas computacionales avanzadas. Si bien estos sistemas sirven como herramientas de apoyo, es imperativo tener en cuenta que el patólogo clínico mantiene la máxima autoridad en la toma de decisiones. La fusión de la experiencia tradicional y la tecnología m, Postprint (published version)

Automotive production systems are designed to produce large quantities in high quality and short throughput times and are therefore organized as line production. This places high quality requirements on the joining processes in automotive body shops, in which automated, robot-guided welding is a key process. The quality of these thermal joining processes depends on various physical and chemical influencing factors, whose interactions cannot be explicitly modelled. This leads to enormous quality assurance efforts in several quality control loops, which may include visual inspections, non-destructive testing of samples to assess the internal structure and destructive testing of samples for the assessment of mechanical properties such as tensile strength. Due to the increasing availability of data in automated processes and the complexity of welding processes, the application of Deep Learning has a great potential to reduce quality control efforts in automotive body shops. Using Deep Learning to leverage process data and accurately predict quality parameters in welding processes is investigated in research, yet model training requires a large, balanced and annotated dataset, whose generation is time and cost intensive, particularly for production data. However, there are generative AI methods such as Generative Adversarial Networks (GANs) that are able to generate synthetic data and thus offer the potential to generate a large amount of annotated production data with relatively little effort. This paper presents a systematic approach to evaluate the potential of incorporating synthetic data in a real-world production dataset to improve quality control using Deep Learning. The approach is validated for the analysis of real-world ultrasound images of resistance spot welding (RSW) processes from the automotive industry. Different Deep Learning architectures to generate synthetic data are compared. Results show that adding synthetic data to the training dataset can improve

The disclosure risk of synthetic/artificial data is still being determined. Studies show that synthetic data generation techniques generate similar data to the original data and sometimes even the exact original data. Therefore, publishing synthetic datasets can endanger the privacy of users. In our work, we study the synthetic data generated from different synthetic data generation techniques, including the most recent diffusion models. We perform a disclosure risk assessment of synthetic datasets via an attribute inference attack, in which an attacker has access to a subset of publicly available features and at least one synthesized dataset, and the aim is to infer the sensitive features unknown to the attacker. We also compute the predictive accuracy and F1 score of the random forest classifier trained on several synthetic datasets. For sensitive categorical features, we show that Attribute Inference Attack is not highly feasible or successful. In contrast, for continuous attributes, we can have an approximate inference. This holds true for the synthetic datasets derived from Diffusion models, GANs, and DPGANs, which shows that we can only have approximated Attribute Inference, not the exact Attribute Inference.

The recent evolution of generative artificial intelligence (GAI) leads to the emergence of groundbreaking applications such as ChatGPT, which not only enhances the efficiency of digital content production, such as text, audio, video, or even network traffic data, but also enriches its diversity. Beyond digital content creation, GAI’s capability in analyzing complex data distributions offers great potential for wireless communications, particularly amidst a rapid expansion of new physical layer communication technologies. For example, the diffusion model can learn input signal distributions and use them to improve the channel estimation accuracy, while the variational autoencoder can model channel distribution and infer latent variables for blind channel equalization. Therefore, this paper presents a comprehensive investigation of GAI’s applications for communications at the physical layer, ranging from traditional issues, including signal classification, channel estimation, and equalization, to emerging topics, such as intelligent reflecting surfaces and joint source channel coding. We also compare GAI-enabled physical layer communications with those supported by traditional AI, highlighting GAI’s inherent capabilities and unique contributions in these areas. Finally, the paper discusses open issues and proposes several future research directions, laying a foundation for further exploration and advancement of GAI in physical layer communications. IEEE

Traffic crash data is often greatly imbalanced with the majority of non-fatal crashes and only a small number of fatal crashes. Such data imbalance issue poses a challenge for crash severity modelling, especially for classifying and interpreting fatal crashes with very limited samples. To address the data imbalance issues, the data resampling techniques are commonly used methods to rebalance the number of samples among all categories of the dataset, such as under-sampling and over-sampling techniques. However, it is challenging for most traditional and existing deep learning-based resampling methods, e.g., synthetic minority oversampling technique (SMOTE) and Generative Adversarial Networks (GAN), to handle both continuous and discrete risk factors in traffic crash datasets, since they are built upon by smooth and continuous functions which are not applicable for processing discrete variables. Though some resampling methods are capable of handling both continuous and discrete variables, they may struggle with mode collapse issues associated with sparse discrete risk factors so that the diversity of the underlying data distribution can not be captured due to oversampling repetitive and similar samples. To address the aforementioned issues, the current study proposes a traffic crash data generation method based on the Conditional Tabular GAN (CTGAN) to rebalance crash datasets for improving performance of crash severity classification and interpretation. The designed experiments are conducted to evaluate contributions of the synthetic data for improving crash severity classification, the distribution consistency between synthetic and benchmark datasets, and the parameter recovery (i.e., the accuracy of parameter estimation and probability prediction) for various resampling strategies. A 4-year real-world dataset collected in Washington State, U.S., and Monte Carlo simulations are utilized for demonstrating the designed experiments. The results indicate that crash seve

A large amount of randomly generated noise in mobile networks leads to a lack of targeting and gaming processes in the speech enhancement process, and the enhancement process from the perspective of acoustic features alone suffers from major drawbacks. Propose a single-channel speech quality enhancement method based on generative adversarial networks in mobile networks. Explain the principle of generative adversarial network to realize single-channel speech quality enhancement in mobile networks and clarify its shortcomings. Design an improved Mel frequency cepstral coefficient extraction method to extract 12 base features as the enhancement basis. Use the relative average least squares loss instead of the traditional loss function to enhance the training efficiency, use the hybrid penalty term to enhance the generator's ability to generate single-channel speech, and optimize the discriminator through the multi-layer convolution and the addition of fully connected layers to enhance the speech quality enhancement ability of adversarial generative networks in various aspects, forming a relative average generative adversarial network (RaGAN) based on hybrid penalty term to realize single-channel speech quality enhancement processing. Through the experiment, when the discriminator is applied with the size of a 3*3 convolutional kernel, the best effect of speech quality enhancement is achieved in the mobile network. This method can complete the enhancement of single-channel speech quality in the mobile network, and the effect is significant, which can effectively reduce the noise in the original single-channel speech.

Automotive production systems are designed to produce large quantities in high quality and short throughput times and are therefore organized as line production. This places high quality requirements on the joining processes in automotive body shops, in which automated, robot-guided welding is a key process. The quality of these thermal joining processes depends on various physical and chemical influencing factors, whose interactions cannot be explicitly modelled. This leads to enormous quality assurance efforts in several quality control loops, which may include visual inspections, non-destructive testing of samples to assess the internal structure and destructive testing of samples for the assessment of mechanical properties such as tensile strength. Due to the increasing availability of data in automated processes and the complexity of welding processes, the application of Deep Learning has a great potential to reduce quality control efforts in automotive body shops. Using Deep Learning to leverage process data and accurately predict quality parameters in welding processes is investigated in research, yet model training requires a large, balanced and annotated dataset, whose generation is time and cost intensive, particularly for production data. However, there are generative AI methods such as Generative Adversarial Networks (GANs) that are able to generate synthetic data and thus offer the potential to generate a large amount of annotated production data with relatively little effort. This paper presents a systematic approach to evaluate the potential of incorporating synthetic data in a real-world production dataset to improve quality control using Deep Learning. The approach is validated for the analysis of real-world ultrasound images of resistance spot welding (RSW) processes from the automotive industry. Different Deep Learning architectures to generate synthetic data are compared. Results show that adding synthetic data to the training dataset can improve

The current research into the issue known as mode collapse has underrepresented the relationship between hyperparameters and mode collapse in GANs. This study aimed to research this further by investigating the effects of four prominent hyperparameters on the manifestation of mode collapse in unconditioned and conditional DCGANs. The investigation was performed via the use of quasi-experiments where the independent variables were the four hyperparameters and the dependent variable was the FID score. Supplemental evaluation was also done by counting classes present with classifiers and visual inspections. The results showed that the number of epochs and the size of the latent dimension only displayed signs of mode collapse at impractically low values. Meanwhile, the learning rate and batch size both had a large effect on mode collapse for higher values. The results showed no significant differences in the hyperparameters’ effect on mode collapse between the unconditioned and conditional DCGANs. The main exception was that the conditional DCGANs were more unstable during training. The conditional DCGANs also occasionally were seen to suffer a phenomenon this study refers to as training failure rebound. This phenomenon had a strong effect on the outputs of the models and is potentially a valuable avenue for future research.

Ultrasound imaging suffers from noise and artifacts that can hinder accurate diagnoses. This project investigates two approaches to address this challenge: Image Processing Algorithm Evaluation and Image Synthesis with Generative Adversarial Networks. The first approach is inspired by low-dose CT reconstruction techniques. We explored the potential of traditional image processing algorithms to improve ultrasound image quality. The performance of various image processing algorithms was evaluated on their ability to enhance image quality using established metrics like signal-to-noise ratio, peak signal-to-noise ratio, contrast-to-noise ratio, structrual similarity index measure, and high-contrast spatial resolution. Our analysis revealed the effectiveness of specific algorithms in different areas of image enhancement, highlighting interesting trends in their performance. For instance, some algorithms excelled at noise reduction, while others demonstrated better detail preservation. For the second approach, considering patient privacy regulations (e.g., GDPR), we investigated the feasibility of synthesizing high-quality ultrasound images using Generative Adversarial Networks. A separate dataset of 10,000 ultrasound images from a phantom was created. The Generative Adversarial Networks architecture was specifically designed to learn the relationship between low-frequency input images and their corresponding high-frequencycounterparts within this dataset. The generated images were evaluated with the same quality metrics used for processed images. Our findings demonstrate the effectiveness of different image processing algorithms in specific areas of image enhancement. We discuss these observations and identify intriguing trends in their performance. Additionally, the Generative Adversarial Networks approach successfully generated high-quality ultrasound images, exhibiting significant improvements in detail preservation and image contrast compared to the low-frequency inp, Ultraljudsundersökning lider av brus och artefakter som kan hindra korrekta diagnoser. Detta projekt undersöker två tillvägagångssätt för att möta denna utmaning: Bildbehandlingsalgoritmutvärdering och bildsyntes med generativa kontradiktoriska nätverk. Det första tillvägagångssättet är inspirerat av lågdos-CT-rekonstruktionstekniker. Vi undersökte potentialen hos traditionella bildbehandlingsalgoritmer för att förbättra ultraljudsbildkvaliteten. Prestandan hos olika bildbehandlingsalgoritmer utvärderades på deras förmåga att förbättra bildkvaliteten med hjälp av etablerade mätvärden som signal-brusförhållande, toppsignal-brusförhållande, kontrast-brusförhållande, strukturellt likhetsindexmått och hög- kontrast spatial upplösning. Vår analys avslöjade effektiviteten hos specifika algoritmer inom olika områden av bildförbättring, vilket lyfte fram intressanta trender i deras prestanda. Vissa algoritmer utmärkte sig till exempel när det gäller brusreducering, medan andra visade bättre detaljbevarande. För det andra tillvägagångssättet, med tanke på patientskyddsbestämmelser (t.ex. GDPR), undersökte vi möjligheten att syntetisera högkvalitativa ultraljudsbilder med hjälp av Generative Adversarial Networks. En separat datauppsättning med 10 000 ultraljudsbilder från en fantom skapades. Arkitekturen Generative Adversarial Networks utformades specifikt för att lära sig förhållandet mellan lågfrekventa ingångsbilder och deras motsvarande högfrekventamotsvarigheter inom denna datauppsättning. De genererade bilderna utvärderades med samma kvalitetsmått som används för bearbetade bilder. Våra resultat visar effektiviteten hos olika bildbehandlingsalgoritmer inom specifika områden av bildförbättring. Vi diskuterar dessa observationer och identifierar spännande trender i deras prestanda. Dessutom genererade tillvägagångssättet Generative Adversarial Networks framgångsrikt högkvalitativa ultraljudsbilder, som uppvisade betydande förbättringar i detaljbevarande och bildkontrast j

The class imbalance of electrocardiogram (ECG) data is a serious impediment to the development of diagnostic systems for heart disease. To address this issue, this paper proposes HQ-DCGAN, a hybrid quantum deep convolutional generative adversarial network, specifically designed for the generation of ECGs. The proposed algorithm employs different quantum convolutional layers for the generator and discriminator as feature extractors and utilizes parameterized quantum circuits (PQCs) to enhance computational capabilities, along with the model-feature mapping process. Moreover, this algorithm preserves the nonlinearity and scalability inherent to classical convolutional neural networks (CNNs), thereby optimizing the utilization of quantum resources, and ensuring compatibility with contemporary quantum devices. In addition, this paper proposes a novel evaluation metric, 1D Fréchet Inception Distance (1DFID), to assess the quality of the generated ECG signals. Simulation experiments show that HQ-DCGAN exhibits strong performance in ECG signal generation. Furthermore, the generated signals achieve an average classification accuracy of 82.2%, outperforming the baseline algorithms. It has been experimentally proven that HQ-DCGAN is friendly to currently noisy intermediate-scale quantum (NISQ) computers, in terms of both number of qubits and circuit depths, while improving the stability. © 2024 The Author(s)

The disclosure risk of synthetic/artificial data is still being determined. Studies show that synthetic data generation techniques generate similar data to the original data and sometimes even the exact original data. Therefore, publishing synthetic datasets can endanger the privacy of users. In our work, we study the synthetic data generated from different synthetic data generation techniques, including the most recent diffusion models. We perform a disclosure risk assessment of synthetic datasets via an attribute inference attack, in which an attacker has access to a subset of publicly available features and at least one synthesized dataset, and the aim is to infer the sensitive features unknown to the attacker. We also compute the predictive accuracy and F1 score of the random forest classifier trained on several synthetic datasets. For sensitive categorical features, we show that Attribute Inference Attack is not highly feasible or successful. In contrast, for continuous attributes, we can have an approximate inference. This holds true for the synthetic datasets derived from Diffusion models, GANs, and DPGANs, which shows that we can only have approximated Attribute Inference, not the exact Attribute Inference.

This study explores the application of Time Series GAN in a Programmable Data Plane (PDP) for enhancing Reinforcement Learning within the context of computer networks, particularly in video applications. We address various challenges, including dataset augmentation, balancing, and extended RL training times in real setups. By leveraging synthetic data generated by TimeGAN, we accelerate experimentation, enhance dataset diversity, and simplify RL model training, ultimately evaluating TimeGAN's performance against real setups in resource optimization for PDPs using an RL agent. This research contributes by directly comparing GAN usage and real setups, bridging a gap in computer network literature, and highlighting a 99% similarity in Quality of Service achieved by an RL model trained with synthetic data, affirming TimeGAN's potential as a valuable simulator without compromising RL training efficacy., Part of ISBN 9798350327939QC 20240725

Ultrasound imaging suffers from noise and artifacts that can hinder accurate diagnoses. This project investigates two approaches to address this challenge: Image Processing Algorithm Evaluation and Image Synthesis with Generative Adversarial Networks. The first approach is inspired by low-dose CT reconstruction techniques. We explored the potential of traditional image processing algorithms to improve ultrasound image quality. The performance of various image processing algorithms was evaluated on their ability to enhance image quality using established metrics like signal-to-noise ratio, peak signal-to-noise ratio, contrast-to-noise ratio, structrual similarity index measure, and high-contrast spatial resolution. Our analysis revealed the effectiveness of specific algorithms in different areas of image enhancement, highlighting interesting trends in their performance. For instance, some algorithms excelled at noise reduction, while others demonstrated better detail preservation. For the second approach, considering patient privacy regulations (e.g., GDPR), we investigated the feasibility of synthesizing high-quality ultrasound images using Generative Adversarial Networks. A separate dataset of 10,000 ultrasound images from a phantom was created. The Generative Adversarial Networks architecture was specifically designed to learn the relationship between low-frequency input images and their corresponding high-frequencycounterparts within this dataset. The generated images were evaluated with the same quality metrics used for processed images. Our findings demonstrate the effectiveness of different image processing algorithms in specific areas of image enhancement. We discuss these observations and identify intriguing trends in their performance. Additionally, the Generative Adversarial Networks approach successfully generated high-quality ultrasound images, exhibiting significant improvements in detail preservation and image contrast compared to the low-frequency inp, Ultraljudsundersökning lider av brus och artefakter som kan hindra korrekta diagnoser. Detta projekt undersöker två tillvägagångssätt för att möta denna utmaning: Bildbehandlingsalgoritmutvärdering och bildsyntes med generativa kontradiktoriska nätverk. Det första tillvägagångssättet är inspirerat av lågdos-CT-rekonstruktionstekniker. Vi undersökte potentialen hos traditionella bildbehandlingsalgoritmer för att förbättra ultraljudsbildkvaliteten. Prestandan hos olika bildbehandlingsalgoritmer utvärderades på deras förmåga att förbättra bildkvaliteten med hjälp av etablerade mätvärden som signal-brusförhållande, toppsignal-brusförhållande, kontrast-brusförhållande, strukturellt likhetsindexmått och hög- kontrast spatial upplösning. Vår analys avslöjade effektiviteten hos specifika algoritmer inom olika områden av bildförbättring, vilket lyfte fram intressanta trender i deras prestanda. Vissa algoritmer utmärkte sig till exempel när det gäller brusreducering, medan andra visade bättre detaljbevarande. För det andra tillvägagångssättet, med tanke på patientskyddsbestämmelser (t.ex. GDPR), undersökte vi möjligheten att syntetisera högkvalitativa ultraljudsbilder med hjälp av Generative Adversarial Networks. En separat datauppsättning med 10 000 ultraljudsbilder från en fantom skapades. Arkitekturen Generative Adversarial Networks utformades specifikt för att lära sig förhållandet mellan lågfrekventa ingångsbilder och deras motsvarande högfrekventamotsvarigheter inom denna datauppsättning. De genererade bilderna utvärderades med samma kvalitetsmått som används för bearbetade bilder. Våra resultat visar effektiviteten hos olika bildbehandlingsalgoritmer inom specifika områden av bildförbättring. Vi diskuterar dessa observationer och identifierar spännande trender i deras prestanda. Dessutom genererade tillvägagångssättet Generative Adversarial Networks framgångsrikt högkvalitativa ultraljudsbilder, som uppvisade betydande förbättringar i detaljbevarande och bildkontrast j

The current research into the issue known as mode collapse has underrepresented the relationship between hyperparameters and mode collapse in GANs. This study aimed to research this further by investigating the effects of four prominent hyperparameters on the manifestation of mode collapse in unconditioned and conditional DCGANs. The investigation was performed via the use of quasi-experiments where the independent variables were the four hyperparameters and the dependent variable was the FID score. Supplemental evaluation was also done by counting classes present with classifiers and visual inspections. The results showed that the number of epochs and the size of the latent dimension only displayed signs of mode collapse at impractically low values. Meanwhile, the learning rate and batch size both had a large effect on mode collapse for higher values. The results showed no significant differences in the hyperparameters’ effect on mode collapse between the unconditioned and conditional DCGANs. The main exception was that the conditional DCGANs were more unstable during training. The conditional DCGANs also occasionally were seen to suffer a phenomenon this study refers to as training failure rebound. This phenomenon had a strong effect on the outputs of the models and is potentially a valuable avenue for future research.

This master’s thesis investigates the classification of microscopic aluminium inclusions using machine learning. Generative Adversarial Networks (GANs) are employed to address the challenge of limited labeled data. The study proposes generating synthetic datasets to improve the robustness of deep learning models in identifying defects in aluminium alloys despite environmental contaminants. The model achieves an impressive overall accuracy of 95.6%, suggesting its potential for improved defect detection. Additionally, the research explores the implications of integrating machine learning and computer vision for the manufacturing sector, particularly in high-strength aluminium alloys where precision is crucial. The findings indicate that this approach can significantly enhance defect detection and classification, leading to more reliable materials for critical applications in aerospace, automotive, and construction industries.

Multiple brain imaging modalities can provide complementary pathologic information for clinical diagnosis. However, it is huge challenge to acquire enough modalities in clinical practice. In this work, a cross-modality reconstruction model, called fine-grain aware generative adversarial network (FA-GAN), is proposed to reconstruct the target modality images of brain from the 2D source modality images with a dual-stages manner. The FA-GAN is able to mine the multi-level shared latent representations from the source modality images and then reconstruct the target modality image from coarse to fine progressively. Specifically, in the coarse stage, the Multi-Grain Extractor firstly extracts and disentangles the shared latent features from the source modality images, and synthesizes the coarse target modality images with a pyramidal network. The Feature-Joint Encoder then encodes the latent features and frequency features jointly. In the fine stage, the Fine-Texture Generator is fed with the joint codes to fine tune the reconstruction of the fine-grained target modality. The wavelet transformation module is employed to extract the frequency codes and guide the Fine-Texture Generator to synthesize finer textures. Comprehensive experiments from MR to PET images on ADNI datasets demonstrate that the proposed model achieves finer structure recovery and outperforms the competing methods quantitatively and qualitatively.

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

In this project, we wish to convert long textual inputs into summarised text chunks and generate images describing the summarized text. This project aims to cultivate a model that can generate true-to-life images from summarized textual input using GAN. GANs aim to estimate and recreate the possible spread of real-world data samples and produce new pictures based on this distribution. This project offers an automated summarised text-to-image synthesis for creating images from written descriptions. The written descriptions serve as the GAN generator's conditional intake. The first step in this synthesis is the use of Natural Language Processing to bring out keywords for summarizing. BART transformers are employed. This is then fed to the GAN network consisting of a generator and discriminator. This project used a pre-trained DALL-E mini model as the GAN architecture.

In this project, we wish to convert long textual inputs into summarised text chunks and generate images describing the summarized text. This project aims to cultivate a model that can generate true-to-life images from summarized textual input using GAN. GANs aim to estimate and recreate the possible spread of real-world data samples and produce new pictures based on this distribution. This project offers an automated summarised text-to-image synthesis for creating images from written descriptions. The written descriptions serve as the GAN generator's conditional intake. The first step in this synthesis is the use of Natural Language Processing to bring out keywords for summarizing. BART transformers are employed. This is then fed to the GAN network consisting of a generator and discriminator. This project used a pre-trained DALL-E mini model as the GAN architecture.

Fast estimation of instantaneous urban airflow distribution can protect pedestrians and objects from high-speed winds and promote high-efficiency natural ventilation. Statistical values based on instantaneous airflow distribution can also provide valuable data for wind environment assessments and urban construction. In this study, we evaluated the potential of generative adversarial networks (GANs), a deep-learning-based generative model that learns the probability distribution of training data and generates new data for estimating instantaneous flow fields in an urban model. It utilizes the velocity obtained from sensors as input, and we investigated the influence of constraints on the estimated results. We used two non-linear models based on GANs: Wasserstein GAN with gradient penalty (WGAN), which has no constraints, and conditional WGAN (CWGAN), which has constraints. We employed a flow field dataset obtained using a large-eddy simulation (LES) for model training and testing. The feasibility and accuracy of the flow field generated by the GANs were verified using the results of the LES and proper orthogonal decomposition–linear stochastic estimation (POD–LSE). For the estimated results, although the WGAN learned the probability distribution of the training data, it was unable to determine the temporal relationship among the estimated data. Conversely, the CWGAN successfully estimated the airflow distribution data for reconstruction and prediction, including the temporal relationship. Furthermore, the statistical quantities of the results from the CWGAN were closer to those of the LES than those of the POD–LSE. © 2023 Elsevier Ltd

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub., Funding Agencies|ITEA/VINNOVA project ASSIST [2021-01420]; LiU Cancer; VINNOVA AIDA [M22-0088]; Ake Wiberg foundation; Wallenberg Center for Molecular Medicine as an associated clinical fellow; [2021-01954]

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub., Funding Agencies|ITEA/VINNOVA project ASSIST [2021-01420]; LiU Cancer; VINNOVA AIDA [M22-0088]; Ake Wiberg foundation; Wallenberg Center for Molecular Medicine as an associated clinical fellow; [2021-01954]

The disclosure risk of synthetic/artificial data is still being determined. Studies show that synthetic data generation techniques generate similar data to the original data and sometimes even the exact original data. Therefore, publishing synthetic datasets can endanger the privacy of users. In our work, we study the synthetic data generated from different synthetic data generation techniques, including the most recent diffusion models. We perform a disclosure risk assessment of synthetic datasets via an attribute inference attack, in which an attacker has access to a subset of publicly available features and at least one synthesized dataset, and the aim is to infer the sensitive features unknown to the attacker. We also compute the predictive accuracy and F1 score of the random forest classifier trained on several synthetic datasets. For sensitive categorical features, we show that Attribute Inference Attack is not highly feasible or successful. In contrast, for continuous attributes, we can have an approximate inference. This holds true for the synthetic datasets derived from Diffusion models, GANs, and DPGANs, which shows that we can only have approximated Attribute Inference, not the exact Attribute Inference.

Wasser, eine lebenswichtige Ressource, steht aufgrund der jahrhundertelangen, weltweiten Verschmutzung durch den Menschen vor noch nie dagewesenen Herausforderungen. Diese weit verbreitete Verschmutzung hat zur Zerstörung der biologischen Vielfalt, zur Verarmung der aquatischen Ökosysteme und zur Kontaminierung der Nahrungskette für Tiere und Menschen geführt. Die Senne, die durch Brüssel, Belgien, fließt, hat unter der jahrhundertelangen, vom Menschen verursachten Wasserverschmutzung gelitten, was zu ihrem biologischen Tod und ihrer Eingeschlossenheit in unterirdischen Tunneln aufgrund von historischen Überschwemmungen und Krankheitsausbrüchen führte. Dieses spekulative Projekt zielt darauf ab, dem Fluss Senne neues Leben einzuhauchen, indem es ihn wieder an die Oberfläche bringt und ein synthetisches, urbanes Ökosystem schafft. Um das Wiederaufleben des Lebens zu erleichtern, muss das Wasser des Flusses von anthropogenen Verunreinigungen und Toxinen gereinigt werden. Im Rahmen dieses Projekts wird die Einführung der Süßwasserperlmuschel, einer Muschelart mit Filterfunktion, als hydrologischer Biofilter vorgeschlagen. Diese Muscheln filtern als "Schirmspezies" verschmutztes Wasser und reichern Schadstoffe in ihren Schalen und Ausscheidungen an. Diese Ökosystemleistung bildet die Grundlage für eine hochwertige Umwelt für Wasserlebewesen und wasserbezogene Arten. Ein umfassendes Verständnis des Verhaltens von Süßwassermuscheln wird durch eine Langzeitstudie gewonnen, bei der drei Exemplare gefilmt und ihre Schalen mikroskopisch untersucht werden. Das "generative adversarial network" StyleGAN wird zum Training auf Muschelschalenstrukturen eingesetzt, um eine morphologische Oberfläche für das Flussbett zu erstellen. Diese künstliche Intelligenz interpretiert Bildeingaben und erzeugt digitale Oberflächen, die die Grundlage für die neue Umgebung der Muschel bilden. Diese neu entstehende synthetische Stadtlandschaft bietet nicht nur optimale Bedingungen für Muscheln, sond, Water, an essential resource for all life, faces unprecedented challenges due to centuries of human-induced pollution on a global scale. This widespread pollution has led to the destruction of biodiversity, depletion of aquatic ecosystems, and contamination of the food chain for both animals and humans. The Senne River, flowing through Brussels, Belgium, has suffered from centuries of human-driven water pollution, resulting in its biological death and confinement to subterranean tunnels due to historical flooding and disease outbreaks. This speculative project aims to breathe new life into the Senne River by bringing it back to the surface and creating a synthetic, urban ecosystem. To facilitate life's resurgence, the river's water must be cleansed of anthropogenic contaminants and toxins. This project proposes the introduction of the Freshwater Pearl Mussel, a bivalve filter-feeding species, as a hydrological biofilter. Acting as an "umbrella species," these mussels filter polluted water, accumulating contaminants in their shells and feces. This ecosystem service lays the groundwork for a high-quality environment for aquatic life and water-related species. A comprehensive understanding of Freshwater Mussel behavior is obtained through a long-term study, including the filming of three specimens, alongside microscopic examinations of their shells. The generative adversarial network, StyleGAN, is employed to train on mussel shell structures, establishing a morphological surface for the riverbed. This artificial intelligence interprets image inputs, generating digital surfaces that form the foundation for the mussel's new environment. This emerging synthetic urban landscape not only provides optimal conditions for mussels but also holds significance for various water-related entities, including the human population of Brussels. The project aims to redefine the river's ecological, social, and cultural importance, altering its ecosystematic significance and fostering rec, Julian Rieder, Masterarbeit Universität Innsbruck 2024

The disclosure risk of synthetic/artificial data is still being determined. Studies show that synthetic data generation techniques generate similar data to the original data and sometimes even the exact original data. Therefore, publishing synthetic datasets can endanger the privacy of users. In our work, we study the synthetic data generated from different synthetic data generation techniques, including the most recent diffusion models. We perform a disclosure risk assessment of synthetic datasets via an attribute inference attack, in which an attacker has access to a subset of publicly available features and at least one synthesized dataset, and the aim is to infer the sensitive features unknown to the attacker. We also compute the predictive accuracy and F1 score of the random forest classifier trained on several synthetic datasets. For sensitive categorical features, we show that Attribute Inference Attack is not highly feasible or successful. In contrast, for continuous attributes, we can have an approximate inference. This holds true for the synthetic datasets derived from Diffusion models, GANs, and DPGANs, which shows that we can only have approximated Attribute Inference, not the exact Attribute Inference.

Histopathological examination, crucial in diagnosing diseases such as cancer, traditionally relies on time- and resource-consuming, poorly standardized chemical staining for tissue visualization. This thesis presents a novel digital alternative using generative neural networks and a point light source (PLS) microscope to transform unstained skin tissue images into their stained counterparts. This approach utilizes PLS-microscopy’s unique illumination angles, providing more structural information about a sample and thereby enhancing a neural network’s ability to produce accurate, virtually stained images. Two matched datasets, each containing paired unstained and chemically stained tissue images, were used for supervised training of several networks. One dataset comprised healthy tissue, while the other, in addition to healthy tissue, included basal and squa- mous cell carcinomas. Given the limited scope of this master’s thesis, which constrained data acquisition, these datasets were relatively small, potentially impacting the general- izability of the model. The project explored the virtual staining capabilities of UNet and DenseUNet architectures, focusing on network depth and input channels. Variations in activation functions, upsampling blocks, and attention gates were tested, alongside the development of Relativistic Generative Adversarial Network (RGAN) models. Quantitative evaluation using standard metrics and qualitative assessment by pathologists and other medical professionals demonstrated the potential of PLS microscopy in virtual staining. The final model, based on RGAN, achieved superior staining accuracy with a structural similarity (SSIM) score of 0.799, significantly outperforming traditional bright field imaging (SSIM 0.631). However, the limited diversity and size of the datasets may have inflated these scores and highlight the need for caution in interpreting the results. The pathologists and medical professionals found virtually stained images ind, In 2023, only 38% of all Swedes diagnosed with cancer began their treatment in time. One reason for this is the time consuming and poorly standardized chemical staining processes. Staining is needed to visualize structures of tissue under a microscope, in order to arrive at a diagnosis. By replacing the chemical staining process with neural networks, a more efficient and sustainable workflow which produces synthetically stained images, referred to as virtual staining, can be achieved. Our thesis presents a novel virtual staining approach using images taken with a PLS microscope as input to the neural networks to try to answer the questions: Can virtual staining performance be improved using PLS images? Do the virtually stained images retain pathologically relevant information?

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Residential load profiles (RLPs) play an increasingly important role in the optimal operation and planning of distribution systems, particularly with the rising integration of low-carbon energy resources such as PV systems, electric vehicles, small-scale batteries, etc. Despite the prevalence of various data-driven models for generating consumption profiles, there is a lack of clear conclusions about their relative strengths and weaknesses. This study undertakes a comprehensive comparison of frequently used data-driven models in recent research, including Generative Adversarial Networks (GANs), Variational Autoencoders (VAE), Wasserstein GANs (WGAN), WGANs with Gradient Penalty (WGANGP), Gaussian Mixture Models (GMMs), and Gaussian Mixture Copulas (GMC). The presented comparison explores the effectiveness of the above-mentioned models on transformer- and consumer-level consumption profiles, as well as for different time resolutions (15-min, 30-min, and 60-min). The objective of this research is to elucidate the respective advantages and drawbacks of these models, thereby providing valuable insights for subsequent research in this field., Intelligent Electrical Power Grids, Electrical Sustainable Energy

Background and objective: Mechanical ventilation is a life-saving treatment for critically-ill patients. During treatment, patient-ventilator asynchrony (PVA) can occur, which can lead to pulmonary damage, complications, and higher mortality. While traditional detection methods for PVAs rely on visual inspection by clinicians, in recent years, machine learning models are being developed to detect PVAs automatically. However, training these models requires large labeled datasets, which are difficult to obtain, as labeling is a labour-intensive and time-consuming task, requiring clinical expertise. Simulating the lung-ventilator interactions has been proposed to obtain large labeled datasets to train machine learning classifiers. However, the obtained data lacks the influence of different hardware, of servo-controlled algorithms, and different sources of noise. Here, we propose VentGAN, an adversarial learning approach to improve simulated data by learning the ventilator fingerprints from unlabeled clinical data. Methods: In VentGAN, the loss functions are designed to add characteristics of clinical waveforms to the generated results, while preserving the labels of the simulated waveforms. To validate VentGAN, we compare the performance for detection and classification of PVAs when training a previously developed machine learning algorithm with the original simulated data and with the data generated by VentGAN. Testing is performed on independent clinical data labeled by experts. The McNemar test is applied to evaluate statistical differences in the obtained classification accuracy. Results: VentGAN significantly improves the classification accuracy for late cycling, early cycling and normal breaths (p < 0.01); no significant difference in accuracy was observed for delayed inspirations (p = 0.2), while the accuracy decreased for ineffective efforts (p < 0.01). Conclusions: Generation of realistic synthetic data with labels by the proposed framework is feasibl

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

Recent advancements in deep learning for aircraft engine fault detection have been predominantly focused on research using simulated datasets. Despite significant progress, the gap between simulated and real-world data underscores a pressing need for models that are more applicable and adaptable to the aerospace industry. This discrepancy stems from factors such as water washes, maintenance activities, noise, and nuanced variations in operating conditions. Further complicating this issue is the lack of failure data leading to class imbalance and limiting the performance of fault classification models. In response to these challenges, this study uses Generative Adversarial Networks (GANs) to augment real-world failure data from General Electric Next Generation (GEnx) aircraft engines. New synthetic data are generated using a Wasserstein GAN with Gradient Penalty (WGAN-GP) and convolutional layers. Evaluation of GAN-generated data remains an active area of research. Accordingly, we also introduce a novel validation method based on a GEnx Gas Path Analysis model. This evaluation step revealed that the GAN could effectively generate gas path response variables that were physically meaningful and consistent with the operating conditions. Furthermore, integrating the GAN-generated data into the original dataset improved the baseline fault detection model’s F1-score by an average of 2.8%. This research also highlights the GAN’s ability to learn and reproduce degradation patterns applicable across different engine units, emphasizing its potential to overcome the challenges between engine unit-to-unit variations. Additionally, this work can potentially be extended to other engine families that require synthetic data to improve maintenance strategies., Aerospace Engineering

In this dissertation, I present two novel graph generative frameworks, the first for code abstract syntax tree (AST) generation from input binary code sequences, and the second for novel texture generation for input 3D meshes. Both frameworks are developed using insights from advances in neural machine translation, in the form of Transformers (multi-headed local and global self-attention), and conditional graph generative neural networks. In the first part of the dissertation, I describe the framework for inferring ASTs from binary sequences. Existing approaches to building decompilers and binary analysis tools are time-consuming and labor-intensive to implement and are not easily adapted to new languages. Existing neural approaches tend to be limited to small code sequences. To address these challenges, we develop a novel and state-of-the-art framework for learning to generate abstract syntax trees from input binary sequences. In the second part of the dissertation, I describe the development of the framework for novel but high-quality texture generation for input 3D meshes. Creating high-quality texture assets for a given 3D mesh model is tedious and time-consuming but has numerous applications in 3D simulation, gaming, and augmented and virtual reality. Existing automated approaches to this problem either require expensive 3D part segmentation or deform the original input mesh. To address these challenges, we develop a new framework that can learn to generate novel textures for 3D mesh models from a collection of 3D meshes and 2D real-world images without any input mesh deformation. Both frameworks are demonstrated to achieve state-of-the-art performance in their respective applications while also providing increased adaptive flexibility.

The development in smart meter technology has made grid operations more efficient based on fine-grained electricity usage data generated at different levels of time granularity. Consequently, machine learning algorithms have benefited from these data to produce useful models for important grid operations. Although machine learning algorithms need historical data to improve predictive performance, these data are not readily available for public utilization due to privacy issues. The existing smart grid data simulation frameworks generate grid data with implicit privacy concerns since the data are simulated from a few real energy consumptions that are publicly available. This paper addresses two issues in smart grid. First, it assesses the level of privacy violation with the individual household appliances based on synthetic household aggregate loads consumption. Second, based on the findings, it proposes two privacy-preserving mechanisms to reduce this risk. Three inference attacks are simulated and the results obtained confirm the efficacy of the proposed privacy-preserving mechanisms.

The development in smart meter technology has made grid operations more efficient based on fine-grained electricity usage data generated at different levels of time granularity. Consequently, machine learning algorithms have benefited from these data to produce useful models for important grid operations. Although machine learning algorithms need historical data to improve predictive performance, these data are not readily available for public utilization due to privacy issues. The existing smart grid data simulation frameworks generate grid data with implicit privacy concerns since the data are simulated from a few real energy consumptions that are publicly available. This paper addresses two issues in smart grid. First, it assesses the level of privacy violation with the individual household appliances based on synthetic household aggregate loads consumption. Second, based on the findings, it proposes two privacy-preserving mechanisms to reduce this risk. Three inference attacks are simulated and the results obtained confirm the efficacy of the proposed privacy-preserving mechanisms.