Search

The availability of comprehensive datasets is a crucial challenge for developing artificial intelligence (AI) models in various applications and fields. The lack of large and diverse public fabric defect datasets forms a major obstacle to properly and accurately developing and training AI models for detecting and classifying fabric defects in real-life applications. Models trained on limited datasets struggle to identify underrepresented defects, reducing their practicality. To address these issues, this study suggests using a conditional generative adversarial network (cGAN) for fabric defect data augmentation. The proposed image-to-image translator GAN features a conditional U-Net generator and a 6-layered PatchGAN discriminator. The conditional U-Network (U-Net) generator can produce highly realistic synthetic defective samples and offers the ability to control various characteristics of the generated samples by taking two input images: a segmented defect mask and a clean fabric image. The segmented defect mask provides information about various aspects of the defects to be added to the clean fabric sample, including their type, shape, size, and location. By augmenting the training dataset with diverse and realistic synthetic samples, the AI models can learn to identify a broader range of defects more accurately. This technique helps overcome the limitations of small or unvaried datasets, leading to improved defect detection accuracy and generalizability. Moreover, this proposed augmentation method can find applications in other challenging fields, such as generating synthetic samples for medical imaging datasets related to brain and lung tumors. [ABSTRACT FROM AUTHOR]

For optimum performance, deep learning methods, such as those applied for retinal and choroidal layer segmentation in optical coherence tomography (OCT) images, require sufficiently large and diverse labelled datasets for training. However, the acquisition and labelling of such data can be difficult or infeasible due to privacy reasons (particularly in the medical domain), accessing patient images such as those with specific pathologies, and the cost and time investment to annotate large volumes of data by clinical experts. Data augmentation is one solution to address this issue, either using simple variations and transformations of the images (e.g. flips, brightness) or using synthetic data from sophisticated generative methods such as generative adversarial networks (GANs). Semi-supervised learning (SSL) is another technique which aims to utilise unlabelled data to enhance the performance of deep learning methods and is beneficial where significant amounts of data may be available but are not labelled. In this study, we aim to enhance patch-based OCT retinal and choroidal layer segmentation with both GAN-based data augmentation and SSL. In particular, we employ a conditional StyleGAN2 to generate synthetic patches for data augmentation and a similar unconditional GAN for pre-training the patch classifier to perform SSL. In doing so, we propose a new patch classifier architecture based on the discriminator architecture to improve performance, in addition to the SSL benefit. Compared to previous methods, the proposed data augmentation approach provides an improved data augmentation performance for patch classification with its effectiveness widespread, particularly in the case of low data, across three different OCT datasets encompassing a range of scanning parameters, noise levels, pathology and participant variability. The method provides some subsequent improvements in boundary delineation which is of high importance from a clinical perspective. Additionally, the proposed SSL approach boosts classification performance and boundary delineation performance in some cases which provides further usefulness in the case of low data. The proposed methods can be utilised to enhance OCT segmentation methods, which may be of considerable benefit for both clinicians and researchers. [ABSTRACT FROM AUTHOR]

Single-image haze removal is an essential preprocessing phase in many object detection and segmentation approaches. Recently, end-to-end deep learning-based approaches have dominated the field of single-image dehazing because of their superiority in recovering clear images corrupted by different types of degradation. However, training an effective dehazing network remains challenging, particularly in the absence of high-quality realistic training datasets. In this paper, a novel approach called a subspace-based dehazing generative adversarial network (SuDGAN) is proposed. Traditional training methods attempt to apply changes to pixel intensities, whereas SuDGAN adopts a novel training approach using existing synthetic datasets to learn the adjustment of subspace components related to haze. This approach enables the network to learn more discriminative haze-aware features and focus on adjusting the components that are more affected by haze (luminance) while preserving those that are less influenced by haze (structure). The proposed SuDGAN, along with several state-of-the-art approaches, is evaluated on various challenging synthetic and realistic datasets using haze-related and traditional evaluation metrics. The experimental results demonstrate the efficiency of SuDGAN in removing haze and producing visually pleasing results. Furthermore, the results show that SuDGAN has clear quantitative and qualitative improvements over most state-of-the-art dehazing approaches. [ABSTRACT FROM AUTHOR]

Contract bridge is a competitive-cooperative multiplayer game. In the bidding phase, the decision-making process is complex, given the extensive range of inaccessible information from the partner and opponents. Inferring partner private information seems to be a viable way to promote cooperation. However, extant research has neglected the reliability and effective utilization of inferred information, consequently limiting adaptability in the face of unpredictable policies and hidden hands of opponents. This paper introduces a novel bidding decision algorithm for contract bridge, utilizing generative adversarial inferring (GAI) and alternate inference-decision reinforcement learning (AID-RL). Specifically, GAI advances the reliability and rationality of partner information inference, factoring in stochastic noises and utilizing adversarial training. Concurrently, AID-RL strengthens the effective and explicit exploitation of inference by integrating a reinforcement objective that increases decision-making payoff into the inference model training alongside the original generative objective. Moreover, a novel self-play approach, augmented by the counterfactual difference reward (CDR) mechanism is proposed to expedite policy updates and ensure robust decision-making against inference biases and unknown opponents. Empirical results demonstrate the superior performance of our algorithm, attaining a noteworthy victory of + 0.359 IMPs against Wbridge5, outperforming EPNN and SiB + 0.596 and + 0.062 IMPs. [ABSTRACT FROM AUTHOR]

The rapid booming of information and communication technologies (ICT) and artificial intelligence has promoted the development of maintenance practices in the building industry towards a data-driven paradigm, of which the digital twin (DT) serves as the fundamental framework to strengthen data transit ability and interoperability. Among the state-of-the-art approaches in the maintenance industry, predictive maintenance (PdM) is a prominent approach by anticipating unexpected failures and unscheduled maintenance tasks. However, most current PdM frameworks are facility-specific, lacking generality and scalability. Besides, existing solutions mainly concentrate on condition monitoring and fault identification rather than failure prediction. Moreover, good prediction results rely heavily on sufficient labelled data sets, which are costly and labour-intensive to collect. To address these issues, the author developed a unified PdM framework for the building industry from the DT perspective. Next, a novel failure prediction method utilising the Semi-supervised Generative Adversarial Network (GAN) has been proposed in this article, which makes effective utilisation of both labelled and unlabelled data. Finally, an online platform has been developed to present the monitoring and prediction information. Experimental findings show the effectiveness and superiority of the proposed method for failure prediction through public data sets of building facilities. [ABSTRACT FROM AUTHOR]

In response to the imbalance between normal and abnormal samples in existing anomaly detection datasets, as well as the complexity in defining anomalies, we introduce a new dataset named Remote Stop to provide data support for existing algorithms. Concurrently, we propose an unsupervised video anomaly detection method based on conditional generative adversarial networks. Our approach trains the model to learn the distribution of normal video data, enabling it to identify anomalous events. The incorporation of a spatial attention mechanism enhances the model's performance in detecting abnormal behaviors in video frames while maintaining high processing efficiency. Moreover, unlike other methods that assess the entire image, our approach uses overlapping image blocks to determine anomalies, enhancing the accuracy and robustness of the model in image segmentation. These innovations not only address the issues of scarce samples and high-cost labeling but also provide new perspectives and tools for video anomaly detection in the field of public safety. The effectiveness of the model was validated on the Avenue and Ped2 datasets and applied to our newly created dataset (Remote Stop), achieving an AUC of 84.3% and processing 61 video frames per second. This enables efficient sequential processing of large-scale video data, offering positive contributions to enhancing public road safety by providing early warnings and enabling timely preventive measures. [ABSTRACT FROM AUTHOR]

In the federated learning scenario, the private data are kept local, and gradients are shared to train the global model. Because gradients are updated according to the private training data, the features of the data are encoded into gradients. Prior work proved the possibility of reconstructing the private training data based on gradients. However, only a small batch of images can be recovered, and the reconstruction quality, especially against the large batch size of images, is unsatisfactory. To improve the quality of reconstruction of a large batch of images, a generative gradient inversion attack based on a regulation term is designed, which is called fDLG. First, a regulation term that can avoid drastic variations within image regions is proposed, which is based on the cognition that changes between image pixels are gradual. The proposed regulation term encourages the synthesized dummy image to be piece-wise smooth. Second, generative adversarial networks are trained to improve the quality of the attack with the global model used as a discriminator. Simulation shows that large batches of images (128 images on CIFAR100, 256 images on MNIST) can be faithfully reconstructed at high resolution, and even large images from ImageNet can be reconstructed. [ABSTRACT FROM AUTHOR]

Generative adversarial networks should produce synthetic data that fits the underlying distribution of the data being modeled. For real-valued time series data, this implies the need to simultaneously capture the static distribution of the data, but also the full temporal distribution of the data for any potential time horizon. This temporal element produces a more complex problem that can potentially leave current solutions under-constrained, unstable during training, or prone to varying degrees of mode collapse. In FETSGAN, entire sequences are translated directly to the generator's sampling space using a seq2seq style adversarial autoencoder, where adversarial training is used to match the training distribution in both the feature space and the lower-dimensional sampling space. This additional constraint provides a loose assurance that the temporal distribution of the synthetic samples will not collapse. In addition, the First Above Threshold operator is introduced to supplement the reconstruction of encoded sequences, which improves training stability and the overall quality of the synthetic data being generated. These novel contributions demonstrate a significant improvement to the current state of the art for adversarial learners in qualitative measures of temporal similarity and quantitative predictive ability of data generated through FETSGAN. [ABSTRACT FROM AUTHOR]

The oil and gas industry employs numerical simulation tools extensively in reservoir analysis and strategic planning. This study presents a machine-learning proxy model, employing a Few-shot Learning approach with a Deep Convolutional Generative Adversarial Network (DC-GAN) to reduce computational costs using fewer training samples. The DC-GAN generates new training samples by synthesizing spatial attributes, reservoir parameters, and time series data, thus enhancing the sample variability and diversity needed for accurate production prediction. Also, the study proposes a straightforward and efficient method for data augmentation that primarily involves replicating the initial training dataset. The accumulated production forecast generated from geostatistical realizations enables intelligent reservoir management through risk analysis. The technique can reduce the processing footprint by 70%. Unlike most reservoir studies that employ synthetic datasets, this investigation adopts a real, high-dimensional, and complex reservoir model. [ABSTRACT FROM AUTHOR]

Image inpainting has made great achievements recently, but it is often tough to generate a semantically consistent image when faced with large missing areas in complex scenes. To address semantic and structural alignment in existing methods for image inpainting, this paper proposes an end-to-end attention-based gated convolution GAN with prior guidance named AGG, which designs the spatial and channel attention mechanisms for full extraction of semantic and structural features. Moreover, AGG constructs the attention-based upsampling module based on the channel attention to refine the feature map and capture more details from features of up-level low sizes. AGG uses the image contour as prior, allowing the gated convolution and attention mechanism may fill the image efficiently by focusing on the contour information. The attention-based gated convolution can effectively capture the global features and compensate for the limitations of the restricted receptive field of the naive convolution. Compared to other models, AGG generates images with finer outline features and no common problems such as the watermark and blur, which shows the best overall performance on the Paris StreetView, CelebA-HQ, and Places2 datasets. The best FID, LPIPS, PSNR, and SSIM values achieved by AGG are 2.18, 0.046, 30.82, and 0.951 on the CelebA-HQ dataset, with at least 3.21% and 6.52% performance improvement on FID and LPIPS compared to state-of-the-art methods, respectively. The source code will be available at https://github.com/Shawn-Yu-1/AGGNet. [ABSTRACT FROM AUTHOR]

This document is a correction notice for an article titled "AGG: attention-based gated convolutional GAN with prior guidance for image inpainting" published in the journal Neural Computing & Applications. The correction states that there was an error in the graphics related to Figures 5, 6, 7, and 8 in the original article. The corrected figures are described briefly, showing comparisons of performance ranking, qualitative comparisons on different datasets, and the best semantic results achieved by the method. The correction notice concludes with a statement from the publisher, Springer Nature, emphasizing their neutrality and impartiality. [Extracted from the article]

Several super-resolution (SR) techniques are introduced in the literature, including traditional and machine learning-based algorithms. Especially, deep learning-based SR approaches emerge with demands for better quality images providing deeper subpixel enhancement. Dealing with the image enhancement task in the satellite images domain, a new SR method for single image SR, namely Enhanced Deep Pyramidal Residual Networks, is introduced in this study. The proposed method overcomes the potential instability problem of Enhanced Deep Residual Networks for Single Image Super-Resolution (EDSR) approach by gradually increasing the feature maps depending upon Pyramidal Residual Networks architecture. The EDSR itself is a good algorithm in the SR domain. However, it has a strict structure for increasing the block size. To overcome this problem with the aim of increasing the algorithm's performance, the pyramidal residual networks gradually increasing hypothesis is utilized in the proposed approach, which is the main contribution and novelty of this study. Besides, by using the pyramidal residual networks gradually increasing hypothesis in the proposed approach, the parameter size of the models is also reduced, which affects the computational time. Two different models are proposed by considering addition and multiplication manners, and the proposed models are evaluated using well-known remote sensing datasets NWPU-RESISC45 and UC Merced. The results obtained by the proposed model are compared with the results of traditional image enhancement algorithms together with the EDSR itself, EDSR with deeper structure, Super-Resolution Generative Adversarial Networks approach, and Residual Local Feature Networks approach in terms of peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM) metrics and showed that the proposed models present better quality images. Moreover, considering the computational time and complexity, it is shown that some proposed models achieve approximately 27% less output parameter having similar PSNR and SSIM values and computational time for EDSR itself and 65% less output parameter having better PSNR and SSIM values and 16% lower computational time for EDSR with deeper structure. [ABSTRACT FROM AUTHOR]

QT interval carries essential information about ventricular depolarization and repolarization. Therefore, in many investigations, it is vital to determine the QT interval accurately and monitor the corresponding QT interval variations. This paper presents a novel implementation of the Generative Adversarial Networks (GAN) to predict the QT waveform interval from the electrocardiogram. The network then estimates the Q and T locations to determine the QT interval. The accuracy of the proposed method in measuring the QT interval was 94.10%. The procedure is evaluated against ground truth performed by expert physicians, and the limits of agreement are used as the measure of performance. The bias in the estimate of the Q locations was 2.92 ms, with 11.48 ms as an upper limit of agreement and − 5.65 ms as the lower limit. The bias in estimating the T location was 1.14 ms with an upper limit of 15.04 ms and a lower limit of − 12.79 ms. Moreover, the bias in the estimate of the QT interval was − 7.10 ms with an upper limit of 50.17 ms and a lower limit of − 64.38 ms. These limits of agreements are clinically acceptable and show that the estimate of the QT interval agrees with the expert's annotations. [ABSTRACT FROM AUTHOR]

The goal of a speech-to-image transform is to produce a photo-realistic picture directly from a speech signal. Current approaches are based on a stacked modular framework that suffers from three vital issues: (1) Training separate networks is time-consuming, inefficient and the convergence of the final generative model depends on the previous generators; (2) The quality of precursor images is ignored; (3) Multiple discriminator networks need to be trained. We propose an efficient and effective single-stage framework called Fusion-S2iGan to yield perceptually plausible and semantically consistent image samples on the basis of spoken descriptions. Fusion-S2iGan introduces a visual+speech fusion module (VSFM), with a pixel-attention module (PAM), a speech-modulation module (SMM) and a weighted-fusion module (WFM), to inject the speech embedding from a speech encoder into the generator while improving the quality of synthesized pictures. The PAM module models the semantic affinities between pixel regions and by assigning larger weights to significant locations. The VSFM module adopts SMM to modulate visual feature maps using fine-grained linguistic cues present in the speech vector. Subsequently, the weighted-fusion model (WFM) captures the semantic importance of the image-attention mask and the speech-modulation module at the level of the channels, in an adaptive manner. Fusion-S2iGan spreads the bimodal information over all layers of the generator network to reinforce the visual feature maps at various hierarchical levels in the architecture. A series of experiments is conducted on four benchmark data sets: CUB birds, Oxford-102, Flickr8k and Places-subset. Results demonstrate the superiority of Fusion-S2iGan compared to the state-of-the-art models with a multi-stage architecture and a performance level that is close to traditional text-to-image approaches. [ABSTRACT FROM AUTHOR]

Grapevine leaf is a commodity that is collected only once a year and has a high return on investment due to its export. However, only certain types of grapevine leaves are consumed. Therefore, it is extremely important to distinguish the types of grapevine leaves. In particular, performing this process automatically on industrial machines will reduce human errors, workload, and thus cost. In this study, a new hybrid approach based on a convolutional neural network is proposed that can automatically distinguish the types of grapevine leaves. In the proposed approach, firstly, the overfitting of network models is prevented by applying data augmentation techniques. Second, new synthetic images were created with the ESRGAN technique to obtain detailed texture information. Third, the top blocks of the MobileNetV2 and VGG19 CNN models were replaced with the newly designed top block, effectively extracting features with the data. Fourthly, the GASVM algorithm was adapted and used to create a subset of the features to eliminate the ineffective and unimportant ones from the obtained features. Finally, SVM classification was performed with the feature subset consisting of 314 features, and approximately 2% higher accuracy and MCC score were obtained compared to the approaches in the literature. [ABSTRACT FROM AUTHOR]

Cardiac image synthesis and analysis are pivotal in modern healthcare for accurate diagnosis and treatment of cardiovascular conditions. This study introduces a pioneering approach—hybrid GAN with semantic resonance—for the generation and analysis of synthetic cardiac images. Drawing from a diverse dataset encompassing local and global views, acquired through various modalities like MRI, CT, and echocardiography, the critical need for both accuracy and clinical relevance in cardiac image synthesis is addressed. In the architectural setup, a hybrid GAN is defined, comprising both local and global components. The local generator excels in capturing intricate details, while the global generator contextualizes the overall cardiac structure. A fundamental innovation lies in the incorporation of pre-trained CNN classifiers specialized in recognizing cardiac pathologies, anatomical structures, and clinical features. These classifiers provide conditional guidance to the generators, ensuring that the synthesized images align semantically with clinical expectations. The training phase employs a pioneering approach, integrating real-time feedback from the classifiers to steer the image synthesis process. The generators, discriminators, and classifiers are optimized, with a dual emphasis on adversarial loss for authenticity and classification loss for diagnostic significance. The accuracy obtained by the proposed approach is 98.96%. Further, the SSIM and PSNR values attained by the proposed approach are 0.955 and 45.23, which is higher than the existing approaches like GAN, CNN, VAE, DenseNet, VGG19, DeepCardiac, Pix2Pix GAN and Cycle-GAN, respectively. Validation on an independent test dataset underscores the generalization capabilities of the proposed hybrid GAN. Furthermore, post-processing techniques refine the generated images, elevating their clinical relevance. Results visualization presents the prowess of the proposed approach, providing a holistic view of local and global perspectives within synthetic images. [ABSTRACT FROM AUTHOR]

The discussion of "can machines think?" which started with the invention of the modern computer, brought along the question of "can machines design?" by researchers in the design field. These developments in information technologies have also affected the architecture. Artificial intelligence applications are encountered in many areas such as pricing estimation, energy conservation security systems of buildings, ventilation systems, user-oriented interactive design solutions, computer-aided programs used in the plan production phase and design process. When the literature on artificial intelligence applications in the architecture is reviewed, it can be seen that it generally includes shape grammars, graph theory, decision trees, constraint-based models, machine learning methods, RNN (Recursive Neural Networks), CNN (Convolutional Neural Network) and GAN (Generative Adversarial Network) algorithms. In this study, the use of artificial intelligence algorithms in architecture was examined, and an example was designed to determine the architectural structures of different periods by using CNN (Convolutional Neural Network). In the study, the open source TensorFlow library developed by Google and the Python programming language were used. Employing a statistical approach and utilizing convolutional neural networks (CNNs), a study has successfully classified the current flow patterns of buildings based on datasets comprising facades of Gothic, Modern, and Deconstructivist architectural styles. The findings demonstrate the efficacy of CNNs in accurately distinguishing the intricate details of diverse architectural styles. Recognizing elements from different periods using the CNN algorithm can examine not only individual buildings but also the relationship of buildings with their environments. It can also gain an important place in the field of conservation of the architectural discipline. The historical processes, aesthetic features and changes of protected buildings can be learned with the CNN algorithm and can guide restoration decisions. As a result of the study, the employed CNN-based model can correctly classify structures with 84.66% accuracy rate. [ABSTRACT FROM AUTHOR]

Nowadays, machine learning (ML) has attained a high level of achievement in many contexts. Considering the significance of ML in medical and bioinformatics owing to its accuracy, many investigators discussed multiple solutions for developing the function of medical and bioinformatics challenges using deep learning (DL) techniques. The importance of DL in Internet of Things (IoT)-based bio- and medical informatics lies in its ability to analyze and interpret large amounts of complex and diverse data in real time, providing insights that can improve healthcare outcomes and increase efficiency in the healthcare industry. Several applications of DL in IoT-based bio- and medical informatics include diagnosis, treatment recommendation, clinical decision support, image analysis, wearable monitoring, and drug discovery. The review aims to comprehensively evaluate and synthesize the existing body of the literature on applying deep learning in the intersection of the IoT with bio- and medical informatics. In this paper, we categorized the most cutting-edge DL solutions for medical and bioinformatics issues into five categories based on the DL technique utilized: convolutional neural network, recurrent neural network, generative adversarial network, multilayer perception, and hybrid methods. A systematic literature review was applied to study each one in terms of effective properties, like the main idea, benefits, drawbacks, methods, simulation environment, and datasets. After that, cutting-edge research on DL approaches and applications for bioinformatics concerns was emphasized. In addition, several challenges that contributed to DL implementation for medical and bioinformatics have been addressed, which are predicted to motivate more studies to develop medical and bioinformatics research progressively. According to the findings, most articles are evaluated using features like accuracy, sensitivity, specificity, F-score, latency, adaptability, and scalability. [ABSTRACT FROM AUTHOR]

Age transformation aims to preserve personalized facial information while altering a given face to appear at a target age. This technique finds extensive applications in fields such as face recognition, movie special effects, and social entertainment, among others. With the advancement of deep learning, particularly Generative Adversarial Networks (GANs), research on age transformation has made significant progress, leading to the emergence of a diverse range of deep learning-based methods. However, a comprehensive and systematic literature review of these methods is currently lacking. In this survey, we provide an all-encompassing review of deep learning methods for facial aging. Firstly, we summarize the key aspects of feature preservation during the age transformation process. Subsequently, we present a comprehensive overview of facial age transformation techniques, categorized according to various deep learning network architectures. Additionally, we conduct an analysis and comparison of commonly used face image datasets, offering recommendations for dataset selection. Furthermore, we consolidate the qualitative and quantitative evaluation metrics commonly employed in age transformation methodologies through experimental assessment. Finally, we address potential areas of future research in age transformation methods, based on the current challenges and limitations. [ABSTRACT FROM AUTHOR]

Knee ligament tear injury is frequent in many volleyball, football, basketball, and cricket players. In the past, various deep learning-based ACL tear detection schemes using knee magnetic resonance imaging (MRI) have been presented. It has shown challenges in ACL tear detection performance due to class imbalance issue arising due to uneven training samples and poor feature representation. This paper presents a simple and effective ACL knee ligament tear detection method with a convolutional neural network (ATD-CNN) to lessen the intricacy of the network. Further, the self-attention mechanism is used to improve the feature representation of MRI image information, neglect the irrelevant information in deep features, and enhance the classification accuracy. To diminish the class imbalance issue, generative adversarial network (GAN) is used to construct the synthetic database. The performance of the ATD-CNN with self-attention is assessed on the MRNet database using precision, accuracy, F1-score, and recall. ATD-CNN provides an accuracy of 90.10% for the original and 93.93% and augmented datasets. However, the ATD-CNN without attention mechanism resulted in 89.60% and 92.30% accuracy for original and augmented dataset. The proposed ATD-CNN model indicates that it can be utilized to detect ACL tears automatically and outperforms the existing schemes for tear detection. [ABSTRACT FROM AUTHOR]

Video-based action recognition faces many challenges, such as complex and varied dynamic motion, spatio-temporal similar action factors, and manual labeling of archived videos over large datasets. How to extract discriminative spatio-temporal action features in videos with resisting the effect of similar factors in an unsupervised manner is pivotal. For that, this paper proposes an unsupervised video-based action recognition method, called two-stream generative adversarial network (TS-GAN), which comprehensively learns the static texture and dynamic motion information inherited in videos with taking the detail information and global information into account. Specifically, the extraction of the spatio-temporal information in videos is achieved by a two-stream GAN. Considering that proper attention to detail is capable of alleviating the influence of spatio-temporal similar factors to the network, a global-detailed layer is proposed to resist similar factors via fusing intermediate features (i.e., detailed action information) and high-level semantic features (i.e., global action information). It is worthwhile of mentioning that the proposed TS-GAN does not require complex pretext tasks or the construction of positive and negative sample pairs, compared with recent unsupervised video-based action recognition methods. Extensive experiments conducted on the UCF101 and HMDB51 datasets have demonstrated that the proposed TS-GAN is superior to multiple classical and state-of-the-art unsupervised action recognition methods. [ABSTRACT FROM AUTHOR]

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that has threatened health worldwide for years, necessitating accurate and rapid diagnostic techniques given its high pathogenicity and transmissibility. Numerous deep learning models have been developed to assist radiologists in chest computed tomography (CT)-based COVID-19 diagnosis. However, existing COVID-19 CT datasets is often characterized by significant geographic and class imbalances, which impede the model's ability to generalize effectively across varying patient cohorts. With the advancements in generative adversarial networks (GAN), one potential solution to the problem is to synthesize data for the target datasets by leveraging a large-scale source dataset for pre-training (i.e., few-shot GAN adaptation). To calibrate the target GAN during adaptation, we incorporate contrastive learning coupled with LeCam regularization, ensuring that the diversity inherent in the source dataset is preserved. Additionally, overfitting is mitigated through the use of consistency regularization and differentiable augmentation techniques. Also, we incorporate off-the-shelf vision models into the discriminator ensemble, predicated on the linear separability between real and fake samples in the feature space, thereby encouraging the generator to match the real distribution in different, complementary feature spaces. We demonstrate the effectiveness of our approach in improving COVID-19 diagnostic performance by generating realistic and diverse CT image-label pairs for the target datasets and show that it consistently outperforms the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper proposes a structure preserving generative adversarial network (S-P GAN) to solve the problem of small structure information loss during the modality translation from computed tomography (CT) to magnetic resonance (MR). A novel generator of the S-P GAN is designed to encode features from CT, where an original CT information branch and its corresponding high-frequency information branch form a dual-branch structure. The small details are highlighted by using a filter in the high-frequency information branch, which offers a complement to the integrity of structural information in CT. Meanwhile, a mixed attention mechanism is introduced to better fuse the dual-branch features for decoding features to MR, where small structure features get more attention in channel and space. Additionally, a new joint loss function is proposed to guide the adversarial training of S-P GAN, which contains structural consistency constrain, pixel translation constrain, and adversarial constrain, so that global similarity and local detail consistency are obtained at the same time. Experimental results show that the results of the S-P GAN are superior to the state-of-the-art models in mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). In real clinical situations, the proposed method has shown good performance in the diagnosis of lumbar disk herniation, the new and old degree of compressibility fracture, and the Modic change of cartilage end plate. [ABSTRACT FROM AUTHOR]

In recent years, the art of film photography has reemerged as a topic of interest for both researchers and the community. Unlike digital photography, which relies on pixels to capture and store information, film photography employs silver halide to capture the scene. This process imbues film photos with a unique colour and textured graininess not present in digital photography. In this paper, we propose Film-GAN, the first Generative Adversarial Network (GAN)-based method for translating digital images to film. Film-GAN generates a corresponding film transformation of the input image based on the desired reference film style. To improve the realism of the generated images, we introduce the colour-noise-encoding (CNE) network, which extracts the colour and graininess of the reference image separately. Our experimental simulations demonstrate that Film-GAN outperforms other state-of-the-art approaches on multiple datasets. Based on evaluations from both professional photographers and amateur photography enthusiasts, the images generated by Film-GAN also received a higher number of votes, indicating its ability to produce better film-effect images. [ABSTRACT FROM AUTHOR]

Alzheimer's disease (AD) is the most frequent type of dementia that has no effective cure, except early discovery and treatment that may help patients to include successful years in patient's lives. Currently, mini-mental state examination (MMSE) score and manual examination of magnetic resource imaging (MRI) scan along with machine learning techniques are used to diagnose the disease; however, they possess certain accuracy limits. Therefore, this paper proposes a deep learning-based multilayered framework for AD classification using transfer learned Alexnet and LSTM for multiclass and binary classification of MR images. However, the deep learning models used in the current study necessitate a large training dataset to produce better outcomes. As a result, this work also utilizes generative adversarial network (GAN) as a data augmentation tool to improve the classification results and further to solve the problem of overfitting. The study uses Alzheimer's disease neuroimaging initiative (ADNI) dataset of 60 AD, 73 mild cognitive impairment (MCI) and 67 cognitively normal (CN) patients from which 2 D MR image scans are extracted. Furthermore, the proposed method achieved the classification accuracy on AD–CN at 98.13%, AD–MCI at 99.38% and CN–MCI at 99.37%, respectively. Also, the multiclass classification shows the promising accuracy of 96.83% for the proposed framework. Finally, the proposed model's performance is compared to other state-of-the-art techniques and the experimental results show that the proposed model outperforms in terms of accuracy, sensitivity and hypothesis testing. [ABSTRACT FROM AUTHOR]

The availability of large-scale facial datasets with the rapid progress of deep learning techniques, such as Generative Adversarial Networks, has enabled anyone to create realistic fake videos. These fake videos can potentially become harmful when used for fake news, hoaxes, and identity fraud. We propose a deep learning bagging ensemble classifier to detect manipulated faces in videos. The proposed bagging classifier uses the convolution and self-attention network (CoAtNet) model as a base learner. CoAtNet model is vertically stacking depthwise convolution layers and self-attention layers in such a way that generalization, capacity, and efficiency are improved. Depthwise convolution captures local features from faces extracted from video then pass these features to the attention layers to extract global information and efficiently capture long-range dependencies of spatial details. Each learner is trained on a different subset randomly taken of training data with a replacement then models' predictions are combined to classify the video either as real or fake. We also use CutMix data augmentation on the extracted faces to enhance the generalization and localization performance of the base learner model. Our experimental results show that our proposed method achieves higher efficiency compared to state-of-the-art methods with AUC values of 99.70%, 97.49%, 98.90%, and 87.62% on the different manipulation techniques of the FaceForensics++ dataset (DeepFakes (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTextures (NT)), respectively, and 99.74% on the Celeb-DF dataset. [ABSTRACT FROM AUTHOR]

The purpose of imbalanced data classification is to solve the problem of unfair learning caused by the large difference in data distribution. Traditional classifiers are designed on the basis of balanced data, but the performance of imbalanced data will decline sharply. Therefore, balancing the majority class and minority class samples before classification is a popular strategy for solving imbalanced learning. Current methods for data balance mainly include oversampling and undersampling. However, the existing undersampling will face the problem of losing important sample information, while oversampling cannot effectively fit the global distribution and generate noise. In recent years, generative adversarial network (GAN) has shown great potential in fitting real sample distributions. Based on this, this paper proposes an improved GAN and biased loss combined model, namely VGAN-BL, to solve the learning problem under imbalanced conditions. In the improvement based on GAN, VAE is used to generate latent vectors with posterior distribution as the input of GAN, and KL similarity measurement loss is introduced into the generator to improve the quality of minority samples generated by GAN. In addition, we propose a biased loss definition method based on the discriminator to improve the performance of classifier. Experiments on 20 real datasets show that the classification performance of the proposed method is significantly improved compared with other advanced methods. The source code can be found here: https://github.com/universuen/VGAN-BL. [ABSTRACT FROM AUTHOR]

Text-to-image generation intends to automatically produce a photo-realistic image, conditioned on a textual description. To facilitate the real-world applications of text-to-image synthesis, we focus on studying the following three issues: (1) How to ensure that generated samples are believable, realistic or natural? (2) How to exploit the latent space of the generator to edit a synthesized image? (3) How to improve the explainability of a text-to-image generation framework? We introduce two new data sets for benchmarking, i.e., the Good & Bad, bird and face, data sets consisting of successful as well as unsuccessful generated samples. This data set can be used to effectively and efficiently acquire high-quality images by increasing the probability of generating Good latent codes with a separate, new classifier. Additionally, we present a novel algorithm which identifies semantically understandable directions in the latent space of a conditional text-to-image GAN architecture by performing independent component analysis on the pre-trained weight values of the generator. Furthermore, we develop a background-flattening loss (BFL), to improve the background appearance in the generated images. Subsequently, we introduce linear-interpolation analysis between pairs of text keywords. This is extended into a similar triangular 'linguistic' interpolation. The visual array of interpolation results gives users a deep look into what the text-to-image synthesis model has learned within the linguistic embeddings. Experimental results on the recent DiverGAN generator, pre-trained on three common benchmark data sets demonstrate that our classifier achieves a better than 98% accuracy in predicting Good/Bad classes for synthetic samples and our proposed approach is able to derive various interpretable semantic properties for the text-to-image GAN model. [ABSTRACT FROM AUTHOR]

Any machine learning (ML) model is highly dependent on the data it uses for learning, and this is even more important in the case of deep learning models. The problem is a data bottleneck, i.e. the difficulty in obtaining an adequate number of cases and quality data. Another issue is improving the learning process, which can be done by actively introducing experts into the learning loop, in what is known as human-in-the-loop (HITL) ML. We describe an ML model based on a neural network in which HITL techniques were used to resolve the data bottleneck problem for the treatment of pancreatic cancer. We first augmented the dataset using synthetic cases created by a generative adversarial network. We then launched an active learning (AL) process involving human experts as oracles to label both new cases and cases by the network found to be suspect. This AL process was carried out simultaneously with an interactive ML process in which feedback was obtained from humans in order to develop better synthetic cases for each iteration of training. We discuss the challenges involved in including humans in the learning process, especially in relation to human–computer interaction, which is acquiring great importance in building ML models and can condition the success of a HITL approach. This paper also discusses the methodological approach adopted to address these challenges. [ABSTRACT FROM AUTHOR]

Today, investigating gait recognition as a biometric technology has become necessary, especially after the COVID-19 pandemic broke out in the world. This paper proposes a deep structure procedure for precise human identification from the walking manner quickly, as each human has a unique way of walking. The proposed generative adversarial network (GAN) generates gait images from pedestrians of the CASIA and OU-ISIR gait datasets for normalizing the number of frames in each class in both datasets. Then, we extract the gait features from the normalized datasets by the pretrained convolutional neural network (CNN) models; AlexNet, Inception, VGG16, VGG19, ResNet, and Xception, and balance them using the synthetic minority oversampling technique (SMOTE) to enhance the recognition results. Several feature selectors are selecting the best features, including particle swarm optimization (PSO), grey wolf optimization (GWO), Chi-square, and genetic models. Finally, the proposed deep neural network recognizes the gait images precisely. Several performance assessment measures were generated to assess the model's quality, including accuracy, precision, sensitivity, specificity, false negative rate (FNR), intersection of union (IoU), and time. Experimental results showed that the inception model with genetic feature selector performed better in all used datasets than the current studies and was more robust against any change, achieving 99.3%, 99.1%, and 99.09% accuracy in identifying CASIA-B, CASIA-A, and OU-ISIR datasets, respectively in low time. [ABSTRACT FROM AUTHOR]

Simulations are crucial for validating the design of engineering systems and their components. However, before simulations can be performed, the geometry of the component must undergo meshing, which involves dividing it into small elements to solve the Partial Differential Equations that describe the phenomenon being simulated. This process can be computationally expensive, so we propose a meshing algorithm that uses Generator Neural Networks to generate refined meshes. Our algorithm is based on a generator used in a Conditional Generative Adversarial Network, which was trained on reference meshes generated by conventional meshing codes. The proposed scheme generated mesh nodal coordinates for both 2D and 3D geometries, and the difference between the element quality of our meshes and the reference meshes used as training data was only 1%. Our algorithm was also 75% faster than the algorithm used to generate the training data meshes. We demonstrate that refined meshes can be generated without relying on computationally expensive solvers. [ABSTRACT FROM AUTHOR]

Facial semantic guidance (including facial landmarks, facial heatmaps, and facial parsing maps) and facial generative adversarial networks (GAN) prior have been widely used in blind face restoration (BFR) in recent years. Although existing BFR methods have achieved good performance in ordinary cases, these solutions have limited resilience when applied to face images with serious degradation and pose-varied (e.g., looking right, looking left, laughing, etc.) in real-world scenarios. In this work, we propose a well-designed blind face restoration network with generative facial prior. The proposed network is mainly comprised of an asymmetric codec and a StyleGAN2 prior network. In the asymmetric codec, we adopt a mixed multi-path residual block (MMRB) to gradually extract weak texture features of input images, which can better preserve the original facial features and avoid excessive fantasy. The MMRB can also be plug-and-play in other networks. Furthermore, thanks to the affluent and diverse facial priors of the StyleGAN2 model, we adopt it as the primary generator network in our proposed method and specially design a novel self-supervised training strategy to fit the distribution closer to the target and flexibly restore natural and realistic facial details. Extensive experiments on synthetic and real-world datasets demonstrate that our model performs superior to the prior art for face restoration and face super-resolution tasks. [ABSTRACT FROM AUTHOR]

Existing Labanotation generation methods have some drawbacks due to low efficiency and incapability to recognize existing videos, which can also be affected by the quality of hardware equipment. To address the issues in existing methods, we propose a new Labanotation generation method for folk dance videos based on pose estimation. Specifically, our method first extracts the key frame images from the fork dance video using temporal differences. Afterward, the 2D joint points of a dancer can be detected from key frame images by using multi-scale fusion of high-resolution net (HRNet), then maps the 2D–3D joint point sequence of the dancer using a pose projection generative adversarial network (pose projection GAN) to predict the coordinates of the 3D joint point position. Finally, the corresponding Labanotation can be generated by analyzing the estimate posture. Experimental results show that the method can achieve the conversion of dance movements in folk dance videos into digital Labanotation, and the automatic generation is much more efficient than manual recording. This method can quickly record endangered folk dances and contribute to the preservation and transmission of movement-based intangible cultural heritage. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) have been widely applied in various domains, including medical image analysis. GANs have been utilized in classification and segmentation tasks, aiding in the detection and diagnosis of diseases and disorders. However, medical image datasets often suffer from insufficiency and imbalanced class distributions. To overcome these limitations, researchers have employed GANs to generate augmented medical images, effectively expanding datasets and balancing class distributions. This review follows the PRISMA guidelines and systematically collects peer-reviewed articles on the development of GAN-based augmentation models. Automated searches were conducted on electronic databases such as IEEE, Scopus, Science Direct, and PubMed, along with forward and backward snowballing. Out of numerous articles, 52 relevant ones published between 2018 and February 2022 were identified. The gathered information was synthesized to determine common GAN architectures, medical image modalities, body organs of interest, augmentation tasks, and evaluation metrics employed to assess model performance. Results indicated that cGAN and DCGAN were the most popular GAN architectures in the reviewed studies. Medical image modalities such as MRI, CT, X-ray, and ultrasound, along with body organs like the brain, chest, breast, and lung, were frequently used. Furthermore, the developed models were evaluated, and potential challenges and future directions for GAN-based medical image augmentation were discussed. This review presents a comprehensive overview of the current state-of-the-art in GAN-based medical image augmentation and emphasizes the potential advantages and challenges associated with GAN utilization in this domain. [ABSTRACT FROM AUTHOR]

This paper addresses the automatic colorization problem, which converts a grayscale image to a colorized one. Recent deep learning approaches can colorize automatically grayscale images. However, when it comes to different scenes which contain distinct color styles, it is difficult to accurately capture the color characteristics. In this work, we propose a fully automatic colorization approach based on Symmetric Positive Definite (SPD) Manifold Learning with a generative adversarial network (SPDGAN) that improves the quality of the colorization results. Our SPDGAN model establishes an adversarial game between two discriminators and a generator. The latter is based on ResNet architecture with few alterations. Its goal is to generate fake colorized images without losing color information across layers through residual connections. Then, we employ two discriminators from different domains. The first one is devoted to the image pixel domain, while the second one is to the Riemann manifold domain which helps to avoid color misalignment. Extensive experiments are conducted on the Places365 and COCO-stuff databases to test the effect of each component of our SPDGAN. In addition, quantitative and qualitative comparisons with state-of-the-art methods demonstrate the effectiveness of our model by achieving more realistic colorized images with less artifacts visually, and good results of PSNR, SSIM, and FID values. [ABSTRACT FROM AUTHOR]

The problem of detecting dangerous or prohibited objects in luggage is a very important step during the implementation of Security setup at Airports, Banks, Government buildings, etc. At present, the most common techniques for detecting such dangerous objects are by using intelligent data analysis algorithms such as deep learning techniques on X-ray imaging or employing a human workforce for inferring the presence of these threat objects in the obtained X-ray images. One of the major challenges while using deep-learning methods to detect such objects is the lack of high-quality threat image data containing the "dangerous" objects (objects of interest) versus the non-threat image data in practical scenarios. So, to tackle this data scarcity problem, anomaly detection techniques using normal data samples have shown great promise. Also, among the available Deep Learning Strategies for anomaly detection for computer vision applications, generative adversarial networks have achieved state-of-the-art results. Considering these insights, we adopted a newly proposed architecture known as Skip-GANomaly and devised a modified version of it by using a UNet++ style generator which performed better than Skip-GANomaly, getting an AUC of 94.94% on Compass-XP, a public X-ray dataset. Finally, for targeting better latent space exploration, we combine these two architectures into an Ensemble, which gives another boost to the performance, getting an AUC of 96.8% on the same Compass-XP, a public X-ray dataset. To further validate the effectiveness of ensemble-based architecture, its performance was tested on patch-based training data on a subset of randomly chosen images of another huge public X-ray dataset named as SIXray, and obtained an AUC of 75.3% on this reduced dataset. To demonstrate the prowess of the discriminator and to bring some explainability to the working of our ensemble, we have used Uniform Manifold Approximation and Projection to plot the latent-space vectors for the dangerous and non-dangerous objects of the test-set; this analysis indicates that the Ensemble learns better features for separating the anomalous class from non-anomalous with respect to the individual architectures. Thus, our proposed architecture provides state-of-the-art results for threat object detection. Most importantly, our models are able to detect threat objects without ever being trained on images containing threat objects. [ABSTRACT FROM AUTHOR]

This study presents a data augmentation technique that solves insufficient/imbalanced data problems during crowdsensing by the Internet of Medical Things (IoMT) or wireless sensor networks (WSNs), owing to diversified locations and heterogeneous conditions. This may cause problems because the samples in various categories may vary in quantities, which create skew distributions. Besides, pattern analysis of insufficient observed samples also generates biased models. In view of such, this work proposes synthetic minority oversampling generative adversarial networks (SMOGANs) for processing imbalanced data, where insufficient samples in quantities can be automatically expanded, so that different classes contain equal numbers of samples, subsequently avoiding biased modeling. The SMOGAN consists of two modules, where the first one is the synthetic minority oversampling technique (SMOTE), and the second involves a GAN. The former is used to initialize the proposed system, in which insufficient/imbalanced data samples are roughly augmented in quantities. Subsequently, the GAN enriches feature diversities of those pseudoreal samples formerly augmented by the SMOTE. Experiments on open datasets were carried out for evaluation. To assess the capability of data augmentation, only 4.00% of the real data were reserved as minority classes and then sent into different data augmentation methods for comparison. Analytical results showed that the proposed SMOGANs outperformed the baselines. Accuracy was increased compared with the baselines. Such results showed that the proposed SMOGAN could improve data collection problems of insufficient/imbalanced datasets by enhancing data quantities and qualities. [ABSTRACT FROM AUTHOR]

Brain Computer Interface is an emerging technology for assisting patients having long term disability. Electroencephalography is the best technique for recording neural activities, however, recording bio signal data from patients is a hectic task. Therefore, Data augmentation using generative adversarial networks is expected to solve this problem. In this paper, we present temporal generative adversarial network with conditional input labels to learn the spatio-temporal features of electroencephalograph and generate realistic data. In order to evaluate the performance of proposed model, we introduce qualitative, quantitative and feature visualization techniques on motor imagery electroencephalograph dataset. Further, we used 1-dimensional temporal convolutional neural network to investigate whether augmented data can replace real data or not for training models. The presented data augmentation framework was compared to three more data augmentation techniques like: copula generative adversarial network, conditional tabular generative adversarial network and tabular variational autoencoder. Our presented technique showed significantly better results to other data augmentation techniques for Electroencephalograph data. The experimental results show that the value of Kullback–Leibler Divergence (KL Divergence) and Inverted Kolmogorov Smirnov D static test (KS test) of data generated by proposed model is the lowest i.e., 0.8940 and 0.7674 respectively. Further, we have tested exhaustively our new data and used a new testing strategy achieving an accuracy of 94.1%. Finally, it is concluded that a time series-based data augmentation technique for Electroencephalograph can reduce long calibration process and thus improve the overall performance of motor imagery user action classification. [ABSTRACT FROM AUTHOR]

Clustering is one of the most fundamental unsupervised learning tasks with numerous applications in various fields. Clustering methods based on neural networks, called deep clustering methods, leverage the representational power of neural networks to enhance clustering performance. ClusterGan constitutes a generative deep clustering method that exploits generative adversarial networks (GANs) to perform clustering. However, it inherits some deficiencies of GANs, such as mode collapse, vanishing gradients and training instability. In order to tackle those deficiencies, the generative approach of implicit maximum likelihood estimation (IMLE) has been recently proposed. In this paper, we present a clustering method based on generative neural networks, called neural implicit maximum likelihood clustering, which adopts ideas from both ClusterGAN and IMLE. The proposed method has been compared with ClusterGAN and other neural clustering methods on both synthetic and real datasets, demonstrating promising results. [ABSTRACT FROM AUTHOR]

Existing deep learning-based image super-resolution (SR) algorithms have made great achievements on synthetic LR–HR pairs in a fully supervised manner. However, the real image degradation process is complicated and unclear, leading to a domain gap between the synthetic LR images and real LR images. To address this issue, the most widely used alternatives are GAN-based methods trained with unpaired data. But most GAN-based methods tend to follow complex multi-stage pipelines and still suffer from the vast cost of hyperparameter tuning on the loss function. In this paper, in contrast to complicated two-stage or multi-stage GAN, a straightforward and simple one-stage GAN-based framework is proposed. Different from the current one-stage GAN, our approach exclusively fits the CycleGAN pattern without any other restrictions, fully leveraging images from unpaired datasets. Specifically, our framework consists of a DegradeNet and an SRNet, which implicitly models the degradation and SR process between the source LR domain and target HR domain. Such a simple one-stage GAN allows us to flexibly learn the data distribution without any assumptions, greatly reducing the model complexity and training difficulty. To further address the inherent practice of empirically setting hyperparameters in previous methods, we propose an adaptive weighting network to automatically learn the contribution weights of different losses in a meta-learning manner. This flexible strategy could shorten the gap between various losses and meanwhile help the model to adaptively optimize in an efficient way. Extensive experiments demonstrate the superiority of our proposed method in terms of indicators and perceptual quality. [ABSTRACT FROM AUTHOR]

Recent advances in digital technologies have lowered the costs and improved the quality of digital pathology Whole Slide Images (WSI), opening the door to apply Machine Learning (ML) techniques to assist in cancer diagnosis. ML, including Deep Learning (DL), has produced impressive results in diverse image classification tasks in pathology, such as predicting clinical outcomes in lung cancer and inferring regional gene expression signatures. Despite these promising results, the uptake of ML as a common diagnostic tool in pathology remains limited. A major obstacle is the insufficient labelled data for training neural networks and other classifiers, especially for new sites where models have not been established yet. Recently, image synthesis from small, labelled datasets using Generative Adversarial Networks (GAN) has been used successfully to create high-performing classification models. Considering the domain shift and complexity in annotating data, we investigated an approach based on GAN that minimized the differences in WSI between large public data archive sites and a much smaller data archives at the new sites. The proposed approach allows the tuning of a deep learning classification model for the class of interest to be improved using a small training set available at the new sites. This paper utilizes GAN with the one-class classification concept to model the class of interest data. This approach minimizes the need for large amounts of labelled data from the new site to train the network. The GAN generates synthesized one-class WSI images to jointly train the classifier with WSIs available from the new sites. We tested the proposed approach for follicular lymphoma data of a new site by utilizing the data archives from different sites. The synthetic images for the one-class data generated from the data obtained from different sites with minimum amount of data from the new site have resulted in a significant improvement of 15% for the Area Under the curve (AUC) for the new site that we want to establish a new follicular lymphoma classifier. The test results have shown that the classifier can perform well without the need to obtain more training data from the test site, by utilizing GAN to generate the synthetic data from all existing data in the archives from all the sites. [ABSTRACT FROM AUTHOR]

In an industrial control system, a programmable logic controller (PLC) plays a vital role in maintaining the stable operation of the system. Cyber-attacks can affect the regular operation by tampering with the data stored in the PLC, thereby damaging to the system. Thus, it is particularly important to develop an efficient cyber-attacks recovery method. However, owing to the impact of unknown factors in theoretical methods, poor scalability of automaton theory, and a lack of constraints during the training process of deep learning network models, the restoration accuracy and stability are low. Therefore, it is a significant challenge to design an appropriate method to improve the accuracy and stability of cyber-attacks recovery. In this study, the generative adversarial networks were applied to the problem of cyber-attacks recovery; furthermore, a multi-stage generative adversarial networks was designed. The model consisted of a Variational Autoencoder and two conditional energy-based generative adversarial networks (CEBGANs). Then the second CEBGAN uses the fitted random noise appending with the data generated by the previous stage and the historical data as additional information to obtain the restoration results. Moreover, a self-adaptive decision policy was established to enhance the restoration accuracy and stability. Experimental results demonstrated that the proposed method in this manuscript could effectively improve the accuracy of cyber-attacks data recovery and reduce the possibility of outliers in data recovery. [ABSTRACT FROM AUTHOR]

Data missing has always occurred in data processing. To solve this problem, researchers have improved the process methods of the missing data with diverse strategies, which range from directly deleting missing data samples to using artificial intelligence technology to filling in incomplete data. The processing methods of the missing data have been improved. Generative adversarial imputation network (GAIN) is a kind of neural network which has an excellent performance in missing data imputation. A number of publications that research and cite the GAIN model show a significant growth trend after GAIN was proposed in 2018. GAIN has been studied and improved by many scholars in their specific fields. However, few studies have systematically surveyed the GAIN model's development trends on missing data from its birth to the present, which result in a lack of comprehensive information about GAINs general performance in different fields. In this review, we summarize the development of the GAIN model in missing data imputation from 2018 to 2022. Based on the WOS database, 32 publications are selected according to the PRISMA statement. The outcome of this paper is from the following aspects: (1) analyzing the publication information and application fields quantitatively; (2) expounding the GAIN-based models, classification, and research trends; (3) elaborating the model attributes and missing data mechanism; and (4) summarizing the existing issues and proposing the future directions. Above all, this paper can help scholars gain further insight into the missing data issues and better understand the optimized directions of GAIN models. [ABSTRACT FROM AUTHOR]

With large amount of training data, Convolutional Neural Network (CNN)-based methods have made great progress for colorizing visible grayscale images. However, existing models rarely consider colorizing near-infrared (NIR) images, which are usually captured with less color or texture information. In this paper, we propose a novel two-stage framework to recover visible color for NIR images with limited data. Firstly, we propose a grayscale preprocessing network to cope with the visual blur and low contrast in NIR gray images, which provides grayscale image as close as the ground truth of visible ones. For the second stage, i.e., image colorization, a novel bilateral Res-Unet network is proposed to transfer color features on both sides of the encoder and decoder to improve the semantic correctness of the colored image. To deal with limited data, a feature memory module is adopted to memorize the color features of training images, providing color conditions for the colorization network. Furthermore, a multi-feature semantic perception loss is reformulated to make the final results with more semantic information and increases the vividness and naturalness. To verify the proposed method, we conduct various experiments on four limited datasets and achieve state-of-the-art against existing CNN-based methods. [ABSTRACT FROM AUTHOR]

Male infertility negatively affects the lives of infertile couples. Sperm head morphology is an essential factor in evaluating semen in male infertility. The lack of samples with abnormal sperm heads compared to the normal sperm head samples makes the classification of sperm head images an imbalance classification problem. In comparison with other deep networks such as convolutional neural networks, capsule neural networks (CapsNets) provide an optimal platform for designing imbalanced classification models by considering spatial relationships of features. Also, generative adversarial networks (GANs) help to improve the imbalanced classification of images by producing suitable synthetic samples. In this paper, a new architecture based on CapsNets and GANs is proposed and evaluated for imbalanced classification of human sperm head images. The new proposed conditional generative adversarial capsule network outperformed other deep learning networks in the balanced and imbalanced classification of human sperm head images. Based on the comparison between the general methods for increasing the data and the proposed network, the general methods have less robustness to reducing the amount of data than the proposed network. The presented network performed a balanced classification of sperm head images with 97.8% accuracy. Additionally, the proposed network maintained an accuracy of over 80% up to the ratio of minority to majority class of 1:30, indicating that it performed properly in the imbalanced classification of sperm images. [ABSTRACT FROM AUTHOR]

In the recent improvement in deep learning approaches for realistic image generation and translation, Generative Adversarial Networks (GANs) delivered favorable results. GAN generates novel samples that look indistinguishable from authentic images. This paper proposes a novel generative network for thermal-to-visible image translation. Thermal to Visible synthesis is challenging due to the non-availability of accurate semantic and textural information in thermal images. The thermal sensors acquire the thermal face images by capturing the object's luminance with fewer details about the actual facial information. However, it is advantageous for low-light and night-time vision, where image information cannot be captured in a complex environment by an RGB camera. We design a new Attention-guided Cyclic Generative Adversarial Network for Thermal to Visible Face transformation (TVA-GAN) by integrating a new attention network. We utilize attention guidance with a recurrent block with an Inception module to simplify the learning space toward the optimum solution. The proposed TVA-GAN is trained and evaluated for thermal to visible face synthesis over three benchmark datasets, including the WHU-IIP, Tufts Face Thermal2RGB, and CVBL-CHILD datasets. The proposed TVA-GAN results show promising improvement in face synthesis compared to the state-of-the-art GAN methods. For the proposed TVA-GAN, code is available at: https://github.com/GANGREEK/TVA-GAN. [ABSTRACT FROM AUTHOR]

The anterior cruciate ligament (ACL) is one of the most commonly injured ligaments in the knee. Accurate tear quantification of ACL plays a crucial role in the treatment of patients. This study aims to propose an auxiliary diagnosis scheme based on deep learning (DL) to assist orthopedic surgeons in automatic ACL tear localization and quantification. The proposed scheme adopted a decoupled generative adversarial network (GAN) to generate the distal residual mask and the normal ACL mask, thereby achieving ACL tear classification. Since the edge information of ACL is important in tear classification, we built the decoupled GAN by decoupling the body and edge parts of masks with different supervision and improved its segmentation performance through a histogram equalization for enhancing image quality, an atrous spatial pyramid pooling (ASPP) module and a distribution module for improving feature representations as well as an effective channel attention mechanism. The experiments showed that the decoupled GAN model achieved promising results in the test set and demonstrated its feasibility in ACL segmentation. The proposed scheme also achieved good results in ACL tear quantification (accuracy: 0.929) and yielded a comparable performance with a senior orthopedic surgeon. This work can provide valuable diagnosis evidence of ACL tear for orthopedic surgeons and is expected to apply in clinical practice. [ABSTRACT FROM AUTHOR]

Layout generation is frequently encountered in the field of mechanical design. The direct application of generative adversarial network, which was originally used to generate pixel-level images, usually cannot guarantee the interrelation between components such as the non-overlap requirement. In addition, the number and the size of components cannot be precisely controlled. These all constitute the characteristics of mechanical layout. To address the above problems, we propose a hierarchical layout generation generative adversarial network (LGGAN) for mechanical layout generation. The layout generator consists of three modules. The first is hierarchical layout generation, where the shape and distribution of components are generated separately using two neural networks. Such a hierarchical structure greatly improves the generation capacity. To reduce the accumulated noise when multiple components are added, a denoiser is included as the second module. The third module is a refinement step used to fine-tune the layouts, which adjusts the size of each component to the prescribed value. All of the three modules are neural network-based, and can be trained through backpropagation. Additionally, an active learning strategy for training the LGGAN is proposed, which allows LGGAN to converge with a small amount of training data in situations where getting a significant amount of training data is not possible. Quantitative and qualitative experiments demonstrate the effectiveness of LGGAN. [ABSTRACT FROM AUTHOR]

This paper presents a new method for detecting the amount of oxide slag on metal ingots, which is a critical indicator of the quality of the ingots. The challenge arises due to the irregular shape and small number of samples of the oxide slag, which makes it difficult to create a pixel-level label dataset for training a semantic segmentation network. To overcome this, a two-stage UNet network based on mixed supervised learning is proposed in this paper. The network uses a combination of a small number of pixel-level labels and a large number of weak label samples to reduce the cost of creating the dataset while achieving better detection accuracy. To solve the issue of overfitting due to the small number of samples, an improved generative adversarial network (GAN) is used to generate oxide slag images with arbitrary shapes to expand the dataset and improve the detection accuracy. Experiments show that the proposed network is highly effective in accurately detecting the distribution of oxide slag on metal ingots. Compared to existing segmentation algorithms, the network achieves higher accuracy while requiring a lower cost for dataset creation. The results of this work have important implications for the practical application of intelligent sensing in slag-picking robots. [ABSTRACT FROM AUTHOR]

Stain normalization is the standardization of the color appearance and has been commonly used in computer-aided diagnosis (CAD) systems. Recently, Generative Adversarial Networks (GANs)-based methods are becoming popular for stain style transfer since they succeed in maintaining structural and color information. The CycleGAN, which is a variant of GANs, has been applied to stain normalization, showing state-of-the-art performance. CycleGAN uses PatchGAN as a discriminator that returns a matrix where each pixel represents the local region of the image. However, the generator utilizes the output of the PatchGAN only for network optimization. The gradient returning to the generator from discriminator is also quite small during training which causes optimizing the network parameters inefficiently. In this paper, we aim to extend the CycleGAN to handle this problem using the output of the convolution layer in the PatchGAN, which we call Regional Realness-Aware Mask, to guide the generator about which regions are incorrect in the input image. In this context, the generator pays more attention to these regions in the next iterations, producing more realistic images. The performance of the proposed Regional Realness-Aware Generative Adversarial Networks (RRAGAN) model was evaluated on a commonly used MITOS-ATYPIA histopathology dataset. The RRAGAN method achieved the highest results in stain normalization compared to the five state-of-the art methods with a large margin in terms of PSNR, SSIM, and RMSE metrics. It achieved PSNR of 24.109, SSIM of 0.933, and RMSE of 8.3576 under the same settings. We also evaluated the impact of the RRAGAN to the segmentation performance on the MICCAI'16 GlaS dataset. It improved the segmentation performance by 4.3% as it reduces the stain color variation. The proposed method could potentially be used as a preprocessing step and can significantly help CAD systems to demonstrate stable performance under color variations. [ABSTRACT FROM AUTHOR]