Search

Sex differences are commonly observed in behaviors that are closely linked to adaptive function, but sex differences can also be observed in behavioral "building blocks" such as locomotor activity and reward processing. Modern neuroscientific inquiry, in pursuit of generalizable principles of functioning across sexes, has often ignored these more subtle sex differences in behavioral building blocks that may result from differences in these behavioral building blocks. A frequent assumption is that there is a default (often male) way to perform a behavior. This approach misses fundamental drivers of individual variability within and between sexes. Incomplete behavioral descriptions of both sexes can lead to an overreliance on reduced "single-variable" readouts of complex behaviors, the design of which may be based on male-biased samples. Here, we advocate that the incorporation of new machine-learning tools for collecting and analyzing multimodal "big behavior" data allows for a more holistic and richer approach to the quantification of behavior in both sexes. These new tools make behavioral description more robust and replicable across laboratories and species, and may open up new lines of neuroscientific inquiry by facilitating the discovery of novel behavioral states. Having more accurate measures of behavioral diversity in males and females could serve as a hypothesis generator for where and when we should look in the brain for meaningful neural differences.

With the widespread prevalence of smart devices, mobile crowdsensing (MCS) becomes a new trend to encourage mobile nodes to participate in cooperative data collection in various Internet of Things (IoT) applications. In location-dependent MCS, location information of mobile nodes is collected and analyzed by service provider to assist in task allocation. If the service provider is not fully trusted, mobile nodes privacy is leaked and accessed by unauthorized parties. How to preserve privacy while maintaining task allocation accuracy and efficiency becomes challenging. To this end, we propose a learning-based mechanism that involves two parts: 1) privacy-preserving task release and task allocation; 2) accurate and efficient task allocation. In the first part, we design a location-based symmetric key generator, which enables two parties to self-generate a symmetric key without depending on fully trusted authorities. By utilizing this key generator and Proxy Re-encryption, we propose a privacy preserving protocol to protect location information in task release and task allocation. In the second part, we design a reinforcement learning based task allocation algorithm to optimize the winners selection, which obtains high accuracy and efficiency. Performance analysis reveals that our proposed mechanism achieves accurate and efficient task allocation while preserving privacy in location-dependent MCS.

Zero-Shot Learning (ZSL) aims to recognize unseen classes that never appear during training. Recently, generative adversarial networks (GANs) have been introduced to convert ZSL into a supervised learning problem by synthesizing unseen visual features. However, since unseen classes are never experienced for the generator during training, the synthesized unseen visual features often become heavily biased towards seen classes, or sometimes there is even no meaningful class that can be assigned to them. This is known as the \textit{bias problem}. In this paper, we propose a novel method, namely Adaptive Bias-Aware GAN (ABA-GAN), to alleviate generating biased visual features. For this purpose, we build a semantic adversarial network to regularize the feature generator. Specifically, an adaptive adversarial loss is proposed to constrain the feature distributions, which avoids the generation of meaningless visual features. Meanwhile, a domain divider is presented to explicitly distinguish synthesized visual features between seen and unseen domains, such that the bias towards seen classes can be alleviated. Moreover, we propose a novel metric named bias score (BS) to explicitly quantify the degree of the strong bias. Extensive experiments on four widely used benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art approaches under both ZSL and GZSL protocols.

For emerging Industrial Internet of Things (IIoT), intelligent anomaly detection is a key step to build smart industry. Specially, explosive time series data brings enormous challenges to the information mining and processing for modern industry. How to identify and detect the multi-dimensional industrial time series anomaly is an important issue. However, most existing studies fail to handle with large amounts of unlabeled data and get the undesirable results. In this paper, we propose a novel integrated deep generative model (DGM), which is built by generative adversarial networks based on bi-directional LSTM and attention mechanism (AMBi-GAN). The structure for generator and discriminator is the bi-directional long short-term memory with attention mechanism (AMBi-LSTM), which can capture time series dependence. Reconstruction loss and generation loss test the input of sample training space and random latent space. Experimental results show that the detection performance of our proposed AMBi-GAN has the potential to improve the detection accuracy of industrial multi-dimensional time series anomaly, towards IIoT in the era of AI.

In recent years, remarkable progress in zero-shot learning (ZSL) has been achieved by generative adversarial networks (GAN). To compensate for the lack of training samples in ZSL, a surge of GAN architectures have been developed by human experts through trial-and-error testing. Despite their efficacy, however, there is still no guarantee that these hand-crafted models can consistently achieve good performance across diversified datasets or scenarios. Accordingly, in this paper, we turn to neural architecture search (NAS) and make the first attempt to bring NAS techniques into the ZSL realm. Specifically, we propose a differentiable GAN architecture search method over a specifically designed search space for zero-shot learning, referred to as ZeroNAS. Considering the relevance and balance of the generator and discriminator, ZeroNAS jointly searches their architectures in a min-max player game via adversarial training. Extensive experiments conducted on four widely used benchmark datasets demonstrate that ZeroNAS is capable of discovering desirable architectures that perform favorably against state-of-the-art ZSL and generalized zero-shot learning (GZSL) approaches. Source code is at https://github.com/caixiay/ZeroNAS.

Thanks to the advancements in deep learning, chatbots are widely used in messaging applications. Undoubtedly, a chatbot is a new way of interaction between humans and machines. However, most of the chatbots act as a simple question answering system that responds with formulated answers. Traditional conversational chatbots usually adopt a retrieval-based model that requires a large amount of conversational data for retrieving various intents. Hence, training a chatbot model that uses low-resource conversational data to generate more diverse dialogues is desirable. We propose a method to build a task-oriented chatbot using a sentence generation model that generates sequences based on the generative adversarial network. The architecture of our model contains a generator that generates a diverse sentence and a discriminator that judges the sentences by comparing the generated and the ground-truth sentences. In the generator, we combine the attention model with the sequence-to-sequence model using hierarchical long short-term memory to extract sentence information. For the discriminator, our reward mechanism assigns low rewards for repeated sentences and high rewards for diverse sentences. Extensive experiments are presented to demonstrate the utility of our model that generates more diverse and information-rich sentences than those of the existing approaches.

Deep Neural Network (DNN) driven technologies have been extensively employed in various aspects of our life. Nevertheless, the applied DNN always fails to detect erroneous behaviors, which may lead to serious problems. Several approaches have been proposed to enhance adversarial examples for automatically testing deep learning (DL) systems, such as image-based DL systems. However, the approaches contain the following two limitations. First, existing approaches only take into account small perturbations on adversarial examples, they design and generate adversarial examples for a certain particular DNN model. This might hamper the transferability of the examples for other DNN models. Second, they only use shallow features (e.g., pixel-level features) to judge the differences between the generated adversarial examples and the original examples. The deep features, which contain high-level semantic information, such as image object categories and scene semantics, are completely neglected. To address these two problems, we propose CAGFuzz, a Coverage-guided Adversarial Generative Fuzzing testing approach for image-based DL systems. CAGFuzz is able to generate adversarial examples for mainstream DNN models to discover their potential errors. First, we train an Adversarial Example Generator (AEG) based on general datasets. AEG only considers the data characteristics to alleviate the transferability problem. Second, we extract the deep features of the original and adversarial examples, and constrain the adversarial examples by cosine similarity to ensure that the deep features of the adversarial examples remain unchanged. Finally, we use the adversarial examples to retrain the models. Based on several standard datasets, we design a set of dedicated experiments to evaluate CAGFuzz. The experimental results show that CAGFuzz can detect more hidden errors, enhance the accuracy of the target DNN models, and generate adversarial examples with higher transferability

In this article, we propose a structure-aligned generative adversarial network framework to improve zero-shot learning (ZSL) by mitigating the semantic gap, domain shift, and hubness problem. The proposed framework contains two parts, i.e., a generative adversarial network with a softmax classifier part, and a structure-aligned part. In the first part, the generative adversarial network aims at generating pseudovisual features through the guiding generator and discriminator play the minimax two-player game together. At the same time, the softmax classifier is committed to increasing the interclass distance and reducing intraclass distance. Then, the harmful effect of domain shift and hubness problems can be mitigated. In another part, we introduce a structure-aligned module where the structural consistency between visual space and semantic space is learned. By aligning the structure between visual space and semantic space, the semantic gap between them can be bridged. The performance of classification is improved when the structure-aligned visual-semantic embedding space is transferred to the unseen classes. Our framework reformulates the ZSL as a standard fully supervised classification task using the pseudovisual features of unseen classes. Extensive experiments conducted on five benchmark data sets demonstrate that the proposed framework significantly outperforms state-of-the-art methods in both conventional and generalized settings.

Ground military target recognition plays a crucial role in unmanned equipment and grasping the battlefield dynamics for military applications, but is disturbed by low-resolution and noisy-representation. In this paper, a recognition method, involving a novel visual attention mechanism-based Gabor region proposal sub-network (Gabor RPN) and improved refinement generative adversarial sub-network (GAN), is proposed. Novel central–peripheral rivalry 3D color Gabor filters are proposed to simulate retinal structures and taken as feature extraction convolutional kernels in low-level layer to improve the recognition accuracy and framework training efficiency in Gabor RPN. Improved refinement GAN is used to solve the problem of blurry target classification, involving a generator to directly generate large high-resolution images from small blurry ones and a discriminator to distinguish not only real images vs. fake images but also the class of targets. A special recognition dataset for ground military target, named Ground Military Target Dataset (GMTD), is constructed. Experiments performed on the GMTD dataset effectively demonstrate that our method can achieve better energy-saving and recognition results when low-resolution and noisy-representation targets are involved, thus ensuring this algorithm a good engineering application prospect.

Model-based approaches to the verification of non-terminating Cyber-Physical Systems (CPSs) usually rely on numerical simulation of the System Under Verification (SUV) model under input scenarios of possibly varying duration, chosen among those satisfying given constraints. Such constraints typically stem from requirements (or assumptions) on the SUV inputs and its operational environment as well as from the enforcement of additional conditions aiming at, e.g., prioritising the (often extremely long) verification activity, by, e.g., focusing on scenarios explicitly exercising selected requirements, or avoiding vacuity in their satisfaction. In this setting, the possibility to efficiently sample at random (with a known distribution, e.g., uniformly) within, or to efficiently enumerate (possibly in a uniformly random order) scenarios among those satisfying the given constraints is a key enabler for the viability of the verification process, e.g., via simulation-based statistical model checking. Unfortunately, in case of non-trivial combinations of constraints, iterative approaches like Markovian random walks in the space of sequences of inputs in general fail in extracting scenarios according to a given distribution, and can be very inefficient to produce legal scenarios of interest. We show how, given a set of constraints on the input scenarios succinctly defined by finite memory monitors, a data structure (scenario generator) can be synthesised, from which any-horizon scenarios satisfying the input constraints can be efficiently extracted by (possibly uniform) random sampling or (randomised) enumeration. Our approach enables seamless support to virtually all simulation-based approaches to CPS verification, ranging from simple random testing to statistical model checking and formal (i.e., exhaustive) verification., Comment: 14 pages. IEEE Transactions on Software Engineering, 2021

Cartoon is a common form of art in our daily life and automatic generation of cartoon images from photos is highly desirable. However, state-of-the-art single-style methods can only generate one style of cartoon images from photos and existing multi-style image style transfer methods still struggle to produce high-quality cartoon images due to their highly simplified and abstract nature. In this article, we propose a novel multi-style generative adversarial network (GAN) architecture, called MS-CartoonGAN, which can transform photos into multiple cartoon styles. MS-CartoonGAN uses only unpaired photos and cartoon images of multiple styles for training. To achieve this, we propose to use (1) a hierarchical semantic loss with sparse regularization to retain semantic content and recover flat shading in different abstract levels, (2) a new edge-promoting adversarial loss for producing fine edges, and (3) a style loss to enhance the difference between output cartoon styles and make training process more stable. We also develop a multi-domain architecture, where the generator consists of a shared encoder and multiple decoders for different cartoon styles, along with multiple discriminators for individual styles. By observing that cartoon images drawn by different artists have their unique styles while sharing some common characteristics, our shared network architecture exploits the common characteristics of cartoon styles, achieving better cartoonization and being more efficient than single-style cartoonization. We show that our multi-domain architecture can theoretically guarantee to output desired multiple cartoon styles. Through extensive experiments including a user study, we demonstrate the superiority of the proposed method, outperforming state-of-the-art single-style and multi-style image style transfer methods.

Cross aging face recognition ability will decrease to recognize someone's face after a certain time. Adding synthetics face images at a certain age generated from face aging architecture is one way to increase the performance of cross aging face recognition. A synthetics face image can create use the Generative Adversarial Network-based architecture. The current Generative Adversarial Network-based in face aging still needs high computation to create a model. Based on that reason we proposed a new optimal variant of Identity Preserving Conditional Generative Adversarial Network (IPCGAN), to generate a synthetic face image at certain age groups. In the proposed architecture, change made at the structure in the generator module, age classification module, and change the objective function to increasing the accuracy performance when generates a realistic synthetic face image in certain age groups and also speed up the training time. Modification in the age classifier at the proposed network forces our architecture to generate better synthetics face in certain age groups. Evaluation using Facenet, and Age prediction shows our method accuracy has 4.2% better results in k-NN classification, 3.6% better accuracy results in SVM classification, 8.6% better accuracy result in age verification, and 4.5% fewer accuracy results in age prediction.

This article investigates the model-free containment control of multiple underactuated unmanned surface vessels (USVs) subject to unknown kinetic models. A novel cooperative control architecture is presented for achieving a containment formation under switching topologies. Specifically, a path-guided distributed containment motion generator (CMG) is first proposed for generating reference points according to the underlying switching topologies. Next, guiding-vector-field-based guidance laws are designed such that each USV can track its reference point, enabling smooth transitions during topology switching. Finally, data-driven neural predictors by utilizing real-time and historical data are developed for estimating total uncertainties and unknown input gains, simultaneously. Based on the learned knowledge from neural predictors, adaptive kinetic control laws are designed and no prior information on kinetic model parameters is required. By using the proposed method, the fleet is able to converge to the convex hull spanned by multiple virtual leaders under switching topologies regardless of fully unknown kinetic models. Through stability analyses, it is proven that the closed-loop control system is input-to-state stable and the tracking errors are uniformly ultimately bounded. Simulation results verify the effectiveness of the proposed cooperative control architecture for multiple underactuated USVs with fully unknown kinetic models.

Text has become one of the most extensively used digital media in Internet, which provides steganography an effective carrier to realize confidential message hiding. Nowadays, generation-based linguistic steganography has made a significant breakthrough due to the progress of deep learning. However, previous methods based on recurrent neural network have two deviations including exposure bias and embedding deviation, which seriously destroys the security of steganography. In this paper, we propose a novel linguistic steganographic model based on adaptive probability distribution and generative adversarial network, which achieves the goal of hiding secret messages in the generated text while guaranteeing high security performance. First, the steganographic generator is trained by using generative adversarial network to effectively tackle the exposure bias, and then the candidate pool is obtained by a probability similarity function at each time step, which alleviates the embedding deviation through dynamically maintaining the diversity of probability distribution. Third, to further improve the security, a novel strategy that conducts information embedding during model training is put forward. We design various experiments from different aspects to verify the performance of the proposed model, including imperceptibility, statistical distribution, anti-steganalysis ability. The experimental results demonstrate that our proposed model outperforms the current state-of-the-art steganographic schemes.

Most of the recent image segmentation methods have tried to achieve the utmost segmentation results using large-scale pixel-level annotated data sets. However, obtaining these pixel-level annotated training data is usually tedious and expensive. In this work, we address the task of semisupervised semantic segmentation, which reduces the need for large numbers of pixel-level annotated images. We propose a method for semisupervised semantic segmentation by improving the confidence of the predicted class probability map via two parts. First, we build an adversarial framework that regards the segmentation network as the generator and uses a fully convolutional network as the discriminator. The adversarial learning makes the prediction class probability closer to 1. Second, the information entropy of the predicted class probability map is computed to represent the unpredictability of the segmentation prediction. Then, we infer the label-error map of the segmentation prediction and minimize the uncertainty on misclassified regions for unlabeled images. In contrast to existing semisupervised and weakly supervised semantic segmentation methods, the proposed method results in more confident predictions by focusing on the misclassified regions, especially the boundary regions. Our experimental results on the PASCAL VOC 2012 and PASCAL-CONTEXT data sets show that the proposed method achieves competitive segmentation performance.

Analog and mixed-signal (AMS) blocks are often a crucial and time-consuming part of System-on-Chip (SoC) design, primarily due to manual circuit and layout iterations. Existing automated solutions for selecting circuit parameters for a given target specification are often not efficient, accurate, or reliable. In order for an automated sizing tool to be practical, we posit that it must: 1) return valid results for a large range of target specifications, 2) understand where and why it is unable to meet certain specifications, 3) consider true layout parasitic simulations for complete end-to-end design, and 4) be automated, allowing most of the design effort to fall on the tool. In this paper, we address these critical points by establishing an automated reinforcement learning framework, AutoCkt, by 1) successfully deploying it on a complex two-stage transimpedance amplifier and two-stage folded-cascode with biasing in 16nm FinFet technology, 2) implementing a new combined distribution deployment algorithm to improve efficiency, 3) analyzing in-depth the efficacy of the trained agent and 4) demonstrating the functionality of this tool when considering a topology that is highly sensitive to layout parasitics. Our algorithm not only successfully reaches unique, valid, and practical performances, but also does so in state-of-the-art run time, up to 38X more efficient than prior work. In addition, our tool averages just four parasitic simulations obtained by using the Berkeley Analog Generator, to achieve a target specification post-layout for the folded-cascode. AutoCkt successfully generates LVS-passed designs with validation in process corner variation results.

This paper proposes a system of stochastic Mixed Complementarity Problems for calculating the Bayesian Nash Equilibrium (BNE) in an electricity spot market. The generators submit strategic multi-level price-offer functions under incomplete information about their rivals marginal cost. This strategic multi-level price offering of each generator is modeled as a standard stochastic bilevel program (BLP) which is then reformulated as its equivalent Mixed Complementarity Problem (MCP). The merger concept is employed and formulated to model multi-level price-offer functions. We then propose the BNE-MCP model which solves the stochastic MCP models of all generators together. A scenario reduction technique is also developed to accurately model the wind and demand uncertainty in the proposed BNE-MCP model. The proposed BNE-MCP model is carefully studied on two illustrative examples and on modified IEEE 14-, 30-, and 57-node systems.

Network embedding aims to learn the low-dimensional node representations for networks, which has attracted an increasing amount of attention in recent years. Most existing efforts in this field attempt to embed the network based on node similarity, which generally relies on edge existence statistics of the network. Instead of relying on the global edge existence statistics for every node pair, in this article, we utilize the information between a pair of nodes in a local way and propose a model, called node pair information preserving network embedding (NINE), based on adversarial networks. The main idea lies in preserving the node pair information (NI) by means of adversarial networks. The architecture of the proposed NINE model consists of three main components, namely: 1) NI embedder; 2) NI generator; and 3) NI discriminator. In the NI embedder, to avoid the complicated similarity calculation for a pair of nodes, the original NI vector calculated from the direct neighbor information of the two nodes is adopted as features, and the edge existence information is taken as labels to learn the embedded NI vector in a supervised learning manner. The second component is the NI generator, which takes the original node representation vectors of a node pair as input and outputs the generated NI vector. In order to make the generated NI vector follow the same distribution of the corresponding embedded NI vector, the generative adversarial network (GAN) is adopted, resulting in the third component, called the NI discriminator. Extensive experiments are conducted on seven real-world datasets in three downstream tasks, namely: 1) network reconstruction; 2) link prediction; and 3) node classification. Comparison results with seven state-of-the-art models demonstrate the effectiveness, efficiency, and rationality of our model.

This paper proposes a parallel recommendation engine, PRECOM, for traffic control operations to mitigate congestion of road traffic in the metropolitan area. The recommendation engine can provide, in real-time, effective and optimal control plans to traffic engineers, who are responsible for manually calibrating traffic signal plans especially when a road network suffers from heavy congestion due to disruptive events. With the idea of incorporating expert knowledge in the operation loop, the PRECOM system is designed to include three conceptual components: an artificial system model, a computational experiment module, and a parallel execution module. Meanwhile, three essential algorithmic steps are implemented in the recommendation engine: a candidate generator based on a graph model, a spatiotemporal ranker, and a context-aware re-ranker. The PRECOM system has been deployed in the city of Hangzhou, China, through both offline and online evaluation. The experimental results are promising, and prove that the recommendation system can provide effective support to the current human-in-the-loop control scheme in the practice of traffic control, operations, and management.

COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy-enhanced COVID-19 detection with generative adversarial networks (GANs) in edge cloud computing. Particularly, we first propose a GAN where a discriminator and a generator based on convolutional neural networks (CNNs) at each edge-based medical institution alternatively are trained to mimic the real COVID-19 data distribution. Then, we propose a new federated learning solution which allows local GANs to collaborate and exchange learned parameters with a cloud server, aiming to enrich the global GAN model for generating realistic COVID-19 images without the need for sharing actual data. To enhance the privacy in federated COVID-19 data analytics, we integrate a differential privacy solution at each hospital institution. Moreover, we propose a new blockchain-based FedGAN framework for secure COVID-19 data analytics, by decentralizing the FL process with a new mining solution for low running latency. Simulations results demonstrate the superiority of our approach for COVID-19 detection over the state-of-the-art schemes., Accepted at IEEE Internet of Things Journal, 14 pages

Modifying facial attributes without the paired dataset proves to be a challenging task. Previously, approaches either required supervision from a ground-truth transformed image or required training a separate model for mapping every pair of attributes. These limit the scalability of the models to accommodate a larger set of attributes since the number of models that we need to train grows exponentially large. Another major drawback of the previous approaches is the unintentional gain of the identity of the person as they transform the facial attributes. We propose a method that allows for controllable and identity-aware transformations across multiple facial attributes using only a single model. Our approach is to train a generative adversarial network (GAN) with a multitask conditional discriminator that recognizes the identity of the face, distinguishes real images from fake, as well as identifies facial attributes present in an image. This guides the generator into producing an output that is realistic while preserving the person’s identity and facial attributes. Through this framework, our model also learns meaningful image representations in a lower dimensional latent space and semantically associate separate parts of the encoded vector with both the person’s identity and facial attributes. This opens up the possibility of generating new faces and other transformations such as making the face thinner or chubbier. Furthermore, our model only encodes the image once and allows for multiple transformations using the encoded vector. This allows for faster transformations since it does not need to reprocess the entire image for every transformation. We show the effectiveness of our proposed method through both qualitative and quantitative evaluations, such as ablative studies, visual inspection, and face verification. Competitive results are achieved compared to the main competition (CycleGAN), however, at great space and extensibility gain by using a single model.

With the increased penetration of renewable sources, power grids are becoming stressed due to fluctuating generation. To alleviate stress from inconsistent sources, utilities employ energy storage systems alongside renewable sources and rely on dispatching synchronous generators. However, optimal dispatch of such devices is limited by traditional AC optimal power flow methods that do not account for time-dependent constraints, such as the state of charge of energy storage systems and generator ramping constraints. Multi-period optimal power flow is proposed as a large non-convex non-linear problem to optimally dispatch and control generators and energy storage elements across multiple time periods. In this paper, we introduce a scalable, robust framework to solve multi-period optimal power flow using a differential dynamic programming scheme that makes it capable of scaling to large systems containing energy storage devices. We demonstrate the efficacy of this solution by optimizing the SyntheticUSA testcase over a set of time periods. A robust homotopy method is applied to achieve fast simulation times that can be parallelized for further improvements.

Textual information appearing on the captured image may contain personal information. In various circumstances, publishing such images in the public domain may create a threat of privacy leak. To avoid these situations, we propose a text concealment method. To accomplish this task, we have used a conditional generator, which is a text region conditioned concealment network. The text regions predicted by the detector network are used as a conditioning criterion for the concealment process. The text region prediction in the form of a word-level bounding box may contain stroke pixels as well as the background pixels. However, to reduce the number of background pixels for proper conditioning of text concealment network, a character level annotation is used for the generator in place of word-level bounding box annotation. It helps to focus more on strokes as compared to the background pixels. A character-level symmetric line representation of text has been proposed to obtain finer level text region prediction as compared to the character-level bounding box. The proposed model is trainable from end-to-end. The text region conditioned generator is trained from the loss of global and local discriminators. The proposed method is validated on public scene text image datasets such as ICDAR 2015, COCO-Text and Synthesis dataset. The proposed architecture shows competitive results as compared to other state-of-the-art approaches.

To boost the object grabbing capability of underwater robots for open-sea farming, we propose a new dataset (UDD) consisting of three categories (seacucumber, seaurchin, and scallop) with 2,227 images. To the best of our knowledge, it is the first 4K HD dataset collected in a real open-sea farm. We also propose a novel Poisson-blending Generative Adversarial Network (Poisson GAN) and an efficient object detection network (AquaNet) to address two common issues within related datasets: the class-imbalance problem and the problem of mass small object, respectively. Specifically, Poisson GAN combines Poisson blending into its generator and employs a new loss called Dual Restriction loss (DR loss), which supervises both implicit space features and image-level features during training to generate more realistic images. By utilizing Poisson GAN, objects of minority class like seacucumber or scallop could be added into an image naturally and annotated automatically, which could increase the loss of minority classes during training detectors to eliminate the class-imbalance problem; AquaNet is a high-efficiency detector to address the problem of detecting mass small objects from cloudy underwater pictures. Within it, we design two efficient components: a depth-wise-convolution-based Multi-scale Contextual Features Fusion (MFF) block and a Multi-scale Blursampling (MBP) module to reduce the parameters of the network to 1.3 million. Both two components could provide multi-scale features of small objects under a short backbone configuration without any loss of accuracy. In addition, we construct a large-scale augmented dataset (AUDD) and a pre-training dataset via Poisson GAN from UDD. Extensive experiments show the effectiveness of the proposed Poisson GAN, AquaNet, UDD, AUDD, and pre-training dataset., Comment: 14 pages, 10 figures

Nuclear atypia scoring (NAS), forms a significant factor in determining individualized treatment plans and also for the prognosis of the disease. Automation of cancer grading using quantitative image-based analysis of histopathological images can circumvent the shortcomings of the prevailing manual grading and can assist the pathologists in cancer diagnosis. However, developing such a robust classifier model require sufficient amount of annotated data, while the labeled histopathological images are scarce and expensive to procure as annotation forms a time-consuming and laborious task. Hence, a semisupervised learning framework combined with the deep neural network based generative adversarial training, that can improve the performance of the classification model with limited annotated data, is proposed in this paper. The proposed NAS-SGAN model consists of discriminator and generator models that are trained in an adversarial manner using both labeled and unlabeled samples. The discriminator model is designed as an unsupervised model stacked over the supervised model sharing the model parameters and learns the data distribution by extracting the discriminative features. The generator model is trained over a stable feature matching objective function following a composite GAN architecture. The novelty of the proposed model is that, we have used a stacked supervised and unsupervised discriminator and a feature matching generator for the NAS-SGAN model and its for the first time the semi-supervised GAN model is explored for the grading of breast cancer. Experimental analysis shows that the proposed model could better discriminate different cancer grades thereby improving the robustness and accuracy of the system, even with limited amount of labeled samples.

This paper proposes a cascading failure simulation (CFS) method and a hybrid machine learning method for vulnerability analysis of integrated power-gas systems (IPGSs). The CFS method is designed to study the propagating process of cascading failures between the two systems, generating data for machine learning with initial states randomly sampled. The proposed method considers generator and gas well ramping, transmission line and gas pipeline tripping, island issue handling and load shedding strategies. Then, a hybrid machine learning model with a combined random forest (RF) classification and regression algorithms is proposed to investigate the impact of random initial states on the vulnerability metrics of IPGSs. Extensive case studies are carried out on three test IPGSs to verify the proposed models and algorithms. Simulation results show that the proposed models and algorithms can achieve high accuracy for the vulnerability analysis of IPGSs.

In recent years, learning-based single image dehazing networks have been comprehensively developed. However, performance improvement is limited due to domain shift between trained synthetic hazy images and untrained real-world hazy images. To alleviate this issue, this paper proposes a real-world dehazing targeted training scheme which nearly realizes paired real-world data training. As a result, a Twofold Multi-scale Generative Adversarial Network (TMS-GAN) consisting of a Haze-generation GAN (HgGAN) and a Haze-removal GAN (HrGAN) is designed. HgGAN attributes real haze properties to synthetic images and HrGAN removes haze from both synthetic and generated fake realistic data under supervision. Thus, the proposed method can better adapt to real-world image dehazing using this cooperative training scheme. Meanwhile, several structural advances of TMS-GAN also improve dehazing performance. Specifically, a haze residual map based on atmospheric scattering model is deduced in HgGAN for fake realistic data generation. The dual-branch generator in HrGAN draws attention to detail restoration by one branch along with another color-branch. A plug-and-play Multi-attention Progressive Fusion Module (MAPFM) is proposed and inserted in both HgGAN and HrGAN. MAPFM incorporates multi-attention mechanism to guide multi-scale feature fusion in a progressive manner, in which Adjacency-attention Block (AAB) can capture contributing features of each level and Self-attention Block (SAB) can establish non-local dependency of feature fusion. Experiments on mainstream benchmarks show that the proposed framework is superior especially on real-world hazy images among single image dehazing methods.

Background: Increasing research reveals that long non-coding RNAs (lncRNAs) play an important role in various biological processes of human diseases. Nonetheless, only a handful of lncRNA-disease associations have been experimentally verified. The study of lncRNA-disease association prediction based on the computational model has provided a preliminary basis for biological experiments to a great degree so as to cut down the huge cost of wet lab experiments. Objective: This study aims to learn the real distribution of lncRNA-disease association from a limited number of known lncRNA-disease association data. This paper proposes a new lncRNA-disease association prediction model called LDA-GAN based on a Generative Adversarial Network (GAN). Method: Aiming at the problems of slow convergence rate, training instabilities, and unavailability of discrete data in traditional GAN, LDA-GAN utilizes the Gumbel-softmax technology to construct a differentiable process for simulating discrete sampling. Meanwhile, the generator and the discriminator of LDA-GAN are integrated to establish the overall optimization goal based on the pairwise loss function. Results: Experiments on standard datasets demonstrate that LDA-GAN achieves not only high stability and high efficiency in the process of confrontation learning but also gives full play to the semisupervised learning advantage of generative adversarial learning framework for unlabeled data, which further improves the prediction accuracy of lncRNA-disease association. Besides, case studies show that LDA-GAN can accurately generate potential diseases for several lncRNAs. Conclusion:: We introduce a generative adversarial model to identify lncRNA-disease associations.

Textures contain a wealth of image information and are widely used in various fields such as computer graphics and computer vision. With the development of machine learning, the texture synthesis and generation have been greatly improved. As a very common element in everyday life, wallpapers contain a wealth of texture information, making it difficult to annotate with a simple single label. Moreover, wallpaper designers spend significant time to create different styles of wallpaper. For this purpose, this paper proposes to describe wallpaper texture images by using multi-label semantics. Based on these labels and generative adversarial networks, we present a framework for perception driven wallpaper texture generation and style transfer. In this framework, a perceptual model is trained to recognize whether the wallpapers produced by the generator network are sufficiently realistic and have the attribute designated by given perceptual description; these multi-label semantic attributes are treated as condition variables to generate wallpaper images. The generated wallpaper images can be converted to those with well-known artist styles using CycleGAN. Finally, using the aesthetic evaluation method, the generated wallpaper images are quantitatively measured. The experimental results demonstrate that the proposed method can generate wallpaper textures conforming to human aesthetics and have artistic characteristics., IEEE Transactions on Circuits and Systems for Video Technology

Wafer map defect recognition (WMDR) is an important part of the integrated circuit manufacturing system. Although deep neural networks (DNNs) have been applied successfully in WMDR, class imbalance and lack of data with class labels affect their performance significantly. A new generative adversarial network (GAN), multi-granularity generative adversarial network (MGGAN), is proposed for wafer map augmentation and enhancement. To alleviate instability and mode collapse of traditional GANs, the lightweight convolution and a two-way information interaction of three subnetworks are considered. MGGAN consists of an auxiliary feature extractor (AFE), a generator (G) and a discriminator (D) for wafer map generation and WMDR. Firstly, a pretrained deep convolutional neural network (CNN), ResNet101, is employed as AFE to extract multi-granularity features from wafer maps. Secondly, in order to improve effectiveness of the adversarial training, a feature matching term is considered in the objective function of the feature generator to minimize the statistical difference between the real images and the generated images. Finally, MGGAN has been successfully applied to WMDR. The experiment results on an industrial dataset WM-811K demonstrate that MGGAN outperforms other typical GANs that aim to solve class imbalance problems and gains better recognition performance than those state-of-the-art DNNs on WMDR. This indicates effectiveness of MGGAN in image enhancement and generation. MGGAN obtains an accuracy of 88.02% on the original data and the pre-trained ResNet101 obtains 93.43% on the data enhanced by MGGAN.

Visible-infrared person re-identification (VI-ReID) is an emerging and challenging cross-modality image matching problem because of the explosive surveillance data in night-time surveillance applications. To handle the large modality gap, various generative adversarial network models have been developed to eliminate the cross-modality variations based on a cross-modal image generation framework. However, the lack of point-wise cross-modality ground-truths makes it extremely challenging to learn such a cross-modal image generator. To address these problems, we learn the correspondence between single-channel infrared images and three-channel visible images by generating intermediate grayscale images as auxiliary information to colorize the single-modality infrared images. We propose a grayscale enhancement colorization network (GECNet) to bridge the modality gap by retaining the structure of the colored image which contains rich information. To simulate the infrared-to-visible transformation, the point-wise transformed grayscale images greatly enhance the colorization process. Our experiments conducted on two visible-infrared cross-modality person re-identification datasets demonstrate the superiority of the proposed method over the state-of-the-arts.

In this letter, we propose a general digital image operation anti-forensic framework based on generative adversarial nets (GANs), called dual-domain generative adversarial network (DDGAN). To tackle the issue of image operation detection, the proposed framework incorporates both operation specific forensic features and machine-learned knowledge to ensure that the generated images exhibit better undetectability performance against various detectors. The DDGAN consists of a generator and two discriminators working on different domains, i.e., the operation-specific feature domain which helps to conceal the artifacts from the perspective of forensic analysis for the target task, and the spatial domain which facilitates to take advantage of machine-learned features from the scratch as a supplementary. Through the experiments on median filtering and JPEG compression anti-forensics, we show the superior performance of the proposed DDGAN compared with state-of-the-art anti-forensic methods in terms of undetectability and visual quality.

As machine learning becomes more popular in the precision engineering sector, the need for large datasets of measurement data increases. Due to the often manual, user dependent and labour-intensive measurement processes, collecting a large amount of data is often infeasible. It would, therefore, be desirable to collect a small amount of data on which to train a model to generate synthetic data that is representative of the real measurement data. To this end, we present an approach to numerical surface texture generation based on a progressively growing generative adversarial network. We show that by encoding height data into grayscale values within an image, the network can create realistic synthetic surface data both qualitatively and quantitatively. The proposed approach is general to any encoded surface; we demonstrate the model trained on two example datasets consisting of surfaces from different manufacturing processes and measured with different techniques. We finally present an extension to the generator model which automatically categorises the produced surfaces, allowing a surface of a desired category to be generated. Finally, we calculate the distributions of areal surface texture parameters for each type of surface and show that there is good agreement between the synthetic and real data.

We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets that share similar structure regularity to facilitate knowledge transfer tasks.

Video moment retrieval has progressed significantly over the past few years, aiming to search the moment that is most relevant to a given natural language query. Most existing methods are trained in a fully-supervised or a weakly-supervised manner, which requires a time-consuming and expensive manually labeling process. In this work, we propose an alternative approach to achieving video moment retrieval that requires no textual annotations of videos and instead leverages the existing visual concept detectors and a pre-trained image-sentence embedding space. Specifically, we design a video-conditioned sentence generator to produce a suitable sentence representation by utilizing the mined visual concepts in videos. We then design a GNN-based relation-aware moment localizer to reasonably select a portion of video clips under the guidance of the generated sentence. Finally, the pre-trained image-sentence embedding space is adopted to evaluate the matching scores between the generated sentence and moment representations with the knowledge transferred from the image domain. By maximizing these scores, the sentence generator and moment localizer can enhance and complement each other to achieve the moment retrieval task. Experimental results on the Charades-STA and ActivityNet Captions datasets demonstrate the effectiveness of our proposed method.

We propose a potential flow generator with L₂ optimal transport regularity, which can be easily integrated into a wide range of generative models, including different versions of generative adversarial networks (GANs) and normalizing flow models. With only a slight augmentation to the original generator loss functions, our generator not only tries to transport the input distribution to the target one but also aims to find the one with minimum L₂ transport cost. We show the effectiveness of our method in several 2-D problems and illustrate the concept of ``proximity'' due to the L₂ optimal transport regularity. Subsequently, we demonstrate the effectiveness of the potential flow generator in image translation tasks with unpaired training data from the MNIST data set and the CelebA data set with a comparison against vanilla Wasserstein GAN with gradient penalty (WGAN-GP) and CycleGAN.

Visualizing the search behavior of a series of points or populations in their native domain is critical in understanding biases and attractors in an optimization process. Distancebased many-objective optimization test problems have been developed to facilitate visualization of search behavior in a two-dimensional design space with arbitrarily many objective functions. Previous works have proposed a few commonly seen problem characteristics into this problem framework, such as the definition of disconnected Pareto sets and dominance resistant regions of the design space. The authors’ previous work has advanced this research further by providing a problem generator to automatically create user-defined problem instances featuring any combination of these problem features as well as newly introduced ones, such as landscape discontinuities, varying objective ranges, and neutrality. This work makes a number of additional contributions including the proposal of an enhanced, open-source feature-rich problem generator that can create user-defined problem instances exhibiting a range of problem features – some of which are newly introduced here or form extensions of existing features. A comprehensive validation of the problem generator is also provided using popular multiobjective optimization algorithms, and some problem generator settings to create instances exhibiting different challenges for an optimizer are identified. peerReviewed

Network embedding (NE) aims to learn low-dimensional node representations of networks while preserving essential node structures and properties. Existing NE methods mainly preserve simple link structures in unsigned networks, neglecting conflicting relationships that widely exist in social media and Internet of things. In this paper, we propose a novel generative adversarial nets learning framework (called SNEGAN) for signed network embedding, which tries to preserve link structures signed by positive or negative labels. SNEGAN combines a generator with a signed random walker technique and a graph softmax function to generate fake links that are used to deceive discriminator. Moreover, it combines a discriminator with a tanh function to discriminate truths and signs of links sampled from the generator and real network structures. The generator and discriminator play a two-player minimax game, and they will eventually generate low-dimensional node representations of signed networks. Extensive experiments on both LFR benchmark and real-world signed networks show the superiority of SNEGAN over the state-of-the-art NE methods in tackling both link (sign) prediction and reconstruction tasks.

Defects, which arise during the growth and processing of the wood, affect the visual and mechanical properties. To improve the utilization of wood with defects, we propose an image inpainting method that reconstructs the texture at defects. The result of image inpainting is visually very consistent with the surrounding healthy board. According to the result, as an application, the plate that is most similar to the surrounding texture could be matched to fill the missing position due to the removal of defects. First, the image inpainting is divided into two stages: coarse inpainting and detailed inpainting. The coarse generator quickly generates the prediction result of the missing area, and the result is fed to the detailed generator to produce a more refined image. Then, the skip connection is applied to the classical Encoder–Decoder in order to solve the problem of information loss due to upsampling. Next, the attention block is introduced in the detailed generator to pay more attention to the meaningful semantics. Finally, normalizing missing and known regions together would result in a bias in the mean and standard deviation, so Partition Normalization was used instead of Batch Normalization. The feature map is divided into two parts, foreground and background, and the two parts are normalized separately. The experimental results show that the results of image inpainting are still high precision with less time spent on the training network. Code, demo and models are available at: https://github.com/NEFUJoeyChen/TG-Net .

Creating realistic characters that can react to the users’ or another character’s movement can benefit computer graphics, games and virtual reality hugely. However, synthesizing such reactive motions in human–human interactions is a challenging task due to the many different ways two humans can interact. While there are a number of successful researches in adapting the generative adversarial network (GAN) in synthesizing single human actions, there are very few on modeling human–human interactions. In this paper, we propose a semi-supervised GAN system that synthesizes the reactive motion of a character given the active motion from another character. Our key insights are two-fold. First, to effectively encode the complicated spatial–temporal information of a human motion, we empower the generator with a part-based long short-term memory (LSTM) module, such that the temporal movement of different limbs can be effectively modeled. We further include an attention module such that the temporal significance of the interaction can be learned, which enhances the temporal alignment of the active–reactive motion pair. Second, as the reactive motion of different types of interactions can be significantly different, we introduce a discriminator that not only tells if the generated movement is realistic or not, but also tells the class label of the interaction. This allows the use of such labels in supervising the training of the generator. We experiment with the SBU, the HHOI and the 2C datasets. The high quality of the synthetic motion demonstrates the effective design of our generator, and the discriminability of the synthesis also demonstrates the strength of our discriminator.

Significant progress has been made with face photo-sketch synthesis in recent years due to the development of deep convolutional neural networks, particularly generative adversarial networks (GANs). However, the performance of existing methods is still limited because of the lack of training data (photo-sketch pairs). To address this challenge, we investigate the effect of knowledge distillation (KD) on training neural networks for the face photo-sketch synthesis task and propose an effective KD model to improve the performance of synthetic images. In particular, we utilize a teacher network trained on a large amount of data in a related task to separately learn knowledge of the face photo and knowledge of the face sketch and simultaneously transfer this knowledge to two student networks designed for the face photo-sketch synthesis task. In addition to assimilating the knowledge from the teacher network, the two student networks can mutually transfer their own knowledge to further enhance their learning. To further enhance the perception quality of the synthetic image, we propose a KD+ model that combines GANs with KD. The generator can produce images with more realistic textures and less noise under the guide of knowledge. Extensive experiments and a user study demonstrate the superiority of our models over the state-of-the-art methods.

Recent proposals of deep learning-based beamformers for ultrasound imaging (US) have attracted significant attention as computational efficient alternatives to adaptive and compressive beamformers. Moreover, deep beamformers are versatile in that image post-processing algorithms can be readily combined. Unfortunately, with the existing technology, a large number of beamformers need to be trained and stored for different probes, organs, depth ranges, operating frequency, and desired target 'styles', demanding significant resources such as training data, etc. To address this problem, here we propose a switchable and tunable deep beamformer that can switch between various types of outputs such as DAS, MVBF, DMAS, GCF, etc., and also adjust noise removal levels at the inference phase, by using a simple switch or tunable nozzle. This novel mechanism is implemented through Adaptive Instance Normalization (AdaIN) layers, so that distinct outputs can be generated using a single generator by merely changing the AdaIN codes. Experimental results using B-mode focused ultrasound confirm the flexibility and efficacy of the proposed method for various applications.

Due to its powerful capability of representation learning and high-efficiency computation, deep hashing has made significant progress in large-scale image retrieval. It has been recognized that deep neural networks are vulnerable to adversarial examples, which is a practical secure problem but seldom studied in deep hashing-based retrieval field. In this paper, we propose a novel prototype-supervised adversarial network (ProS-GAN), which formulates a flexible generative architecture for efficient and effective targeted hashing attack. To the best of our knowledge, this is one of the first generation-based methods to attack deep hashing networks. Generally, our proposed framework consists of three parts, i.e., a PrototypeNet, a Generator and a Discriminator. Specifically, the designed PrototypeNet embeds the target label into the semantic representation and learns the prototype code as the category-level representative of the target label. Moreover, the semantic representation and the original image are jointly fed into the generator for flexible targeted attack. Particularly, the prototype code is adopted to supervise the generator to construct the targeted adversarial example by minimizing the Hamming distance between the hash code of the adversarial example and the prototype code. Furthermore, the generator fools the discriminator to simultaneously encourage the adversarial examples visually realistic and the semantic representation informative. Extensive experiments demonstrate that the proposed framework can efficiently produce adversarial examples with better targeted attack performance and transferability over state-of-the-art targeted attack methods of deep hashing.

Overlooked issues in the selection and presentation of name generators may lead to extreme measurement errors in Web surveys. This paper compares networks elicited from the standard “important matters” (IM) name generator, which records alter names on one or five boxes per page, with a two-generator design and an alternative approach, the R5D. The R5D consists of five name generators focused on a range of topics commonly discussed as important matters. Each generator is restricted to recording one alter. Generally accepted practices in egocentric surveys result in inaccurate measures of network size and composition. They include the use of the stand-alone IM generator and advice to present a single name box to record one alter name per page. The use of a single name box encourages satisficing. The R5D is a parsimonious alternative with stronger construct validity. It has less measurement error, provides measures of social support, and enumerates alters with less slippage in importance than other approaches.

In this paper, we propose a novel framework to translate a portrait photo-face into an anime appearance. Different from existing translation methods which do not designate specific styles, we aim to synthesize anime-faces which are style-consistent with a given reference anime-face. However, unlike typical translation tasks, such anime-face translation is particularly challenging due to the large and complex variations of appearances among anime-faces. Existing methods often fail to transfer the styles of reference anime-faces to the generated anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated anime-faces. We propose a novel GAN-based anime-face translator, called AniGAN, to synthesize high-quality anime-faces. Specifically, a new generator architecture is proposed to simultaneously transfer color/texture styles and transform local facial shapes into anime-like counterparts based on the style of a reference anime-face, while preserving the global structure of the source photo-face. New normalization functions are designed for the generator to further improve local shape transformation and color/texture style transfer. Besides, we propose a double-branch discriminator to learn domain-specific distributions through individual branches and learn cross-domain shared distributions via shared layers, helping generate visually pleasing anime-faces and effectively mitigate artifacts/distortions. Extensive experiments on benchmark datasets qualitatively and quantitatively demonstrate the superiority of our method over state-of-the-art methods.

Missing data reconstruction is a classical yet challenging problem in remote sensing images. Most current methods based on traditional convolutional neural network require supplementary data and can only handle one specific task. To address these limitations, we propose a novel generative adversarial network-based missing data reconstruction method in this letter, which is capable of various reconstruction tasks given only single source data as input. Two auxiliary patch-based discriminators are deployed to impose additional constraints on the local and global regions, respectively. In order to better fit the nature of remote sensing images, we introduce special convolutions and attention mechanism in a two-stage generator, thereby benefiting the tradeoff between accuracy and efficiency. Combining with perceptual and multiscale adversarial losses, the proposed model can produce coherent structure with better details. Qualitative and quantitative experiments demonstrate the uncompromising performance of the proposed model against multisource methods in generating visually plausible reconstruction results. Moreover, further exploration shows a promising way for the proposed model to utilize spatio-spectral-temporal information. The codes and models are available at https://github.com/Oliiveralien/Inpainting-on-RSI.

Cross-domain face synthesis plays a positive role in law enforcement and digital entertainment. It is challenging to synthesize high-quality faces across multiple domains based on limited paired data because the multiple mappings between different domains may interfere with each other. Cognitive science investigates that the brain can recognize the same person with multiple different expressions by extracting invariant information on the face and we humans perceive instances by decomposing them into parts. Motivated by these cognitions, we propose a unified semi-supervised framework MFS for multi-domain face synthesis by extracting a domain-invariant representation and exploiting parts of multi-level features. Specifically, realized by adversarial training with additional ability to utilize domain-specific information, a encoder is trained to remove domain specific information and extract the domain-invariant representation from multiple inputs. Then, we utilize the multi-level feature parts extracted from inputs and reconstructed faces via a pretrained recognition model to ensure that the domain-invariant representation contains enough semantic information. we also utilize the feature parts extracted from inputs and limited paired data to compose pseudo features in target domain for supervising the generator, which makes our framework suitable for large amounts of unpaired training data. By exploiting this framework, we can achieve face synthesis between multiple domains using some paired data together with a large training database without ground truth target faces. Experimental results demonstrate our framework achieves great performances on qualitative and quantitative evaluations under both artificial and uncontrolled environments, and our framework has competitive performances in single translation compared with specialized methods for translation between two specific domains.

In zero-shot learning (ZSL) tasks, especially in generalized zero-shot learning (GZSL), the model tends to classify unseen test samples into seen categories, which is well known as the domain shift problem, because the model is trained from seen samples without unseen samples. Recently, generative adversarial network (GAN) based methods have achieved good performance in GZSL, which replace real unseen features by synthesizing fake ones to mitigate the domain shift. However, the domain shift problem is still not well solved, due to the lacking of unseen samples in the training progress of the GAN generator. In this paper, we propose a generative model named discriminative learning GAN (DL-GAN) to alleviate the domain shift in GZSL. Specifically, the DL-GAN is designed with three novel components: a dual-stream embedding model that aligns features to the ground-truth attributes to extract discriminative latent attributes from features, an attribute-based generative model that generates high-quality unseen features from semantic attributes to guarantee inter-class discriminability and semantic consistency, and a seen/unseen classifier that leverages validation samples to distinguish seen samples from unseen ones. Experimental results on four widely used datasets verify that our proposed approach significantly outperforms the state-of-the-art methods under the GZSL protocol.

Fusing multi-modality medical images, such as MR and PET, can provide various anatomical or functional information about human body. But PET data is always unavailable due to different reasons such as cost, radiation, or other limitations. In this paper, we propose a 3D end-to-end synthesis network, called Bidirectional Mapping Generative Adversarial Networks (BMGAN), where image contexts and latent vector are effectively used and jointly optimized for brain MR-to-PET synthesis. Concretely, a bidirectional mapping mechanism is designed to embed the semantic information of PET images into the high dimensional latent space. And the 3D DenseU-Net generator architecture and the extensive objective functions are further utilized to improve the visual quality of synthetic results. The most appealing part is that the proposed method can synthesize the perceptually realistic PET images while preserving the diverse brain structures of different subjects. Experimental results demonstrate that the performance of the proposed method outperforms other competitive cross-modality synthesis methods in terms of quantitative measures, qualitative displays, and classification evaluation., 12pages, 10 figures

Generative Adversarial Networks (GANs) are capable of effectively synthesising new realistic images and estimating the potential distribution of samples utilising adversarial learning. Nevertheless, conventional GANs require a large amount of training data samples to produce plausible results. Inspired by the capacity for humans to quickly learn new concepts from a small number of examples, several meta-learning approaches for the few-shot datasets are presented. However, most of meta-learning algorithms are designed to tackle few-shot classification and reinforcement learning tasks. Moreover, the existing meta-learning models for image generation are complex, thereby affecting the length of training time required. Fast Adaptive Meta-Learning (FAML) based on GAN and the encoder network is proposed in this study for few-shot image generation. This model demonstrates the capability to generate new realistic images from previously unseen target classes with only a small number of examples required. With 10 times faster convergence, FAML requires only one-fourth of the trainable parameters in comparison baseline models by training a simpler network with conditional feature vectors from the encoder, while increasing the number of generator iterations. The visualisation results are demonstrated in the paper. This model is able to improve few-shot image generation with the lowest FID score, highest IS, and comparable LPIPS to MNIST, Omniglot, VGG-Faces, and miniImageNet datasets.