Search

Conditional generative adversarial networks (cGANs) have demonstrated remarkable success due to their class-wise controllability and superior quality for complex generation tasks. Typical cGANs solve the joint distribution matching problem by decomposing two easier sub-problems: marginal matching and conditional matching. In this paper, we proposes a simple but effective training methodology, selective focusing learning, which enforces the discriminator and generator to learn easy samples of each class rapidly while maintaining diversity. Our key idea is to selectively apply conditional and joint matching for the data in each mini-batch.Specifically, we first select the samples with the highest scores when sorted using the conditional term of the discriminator outputs (real and generated samples). Then we optimize the model using the selected samples with only conditional matching and the other samples with joint matching. From our toy experiments, we found that it is the best to apply only conditional matching to certain samples due to the content-aware optimization of the discriminator. We conducted experiments on ImageNet ( $64 \times 64$ and $128 \times 128$ ), CIFAR-10, CIFAR-100 datasets, and Mixture of Gaussian, noisy label settings to demonstrate that the proposed method can substantially (up to 35.18% in terms of FID) improve all indicators with 10 independent trials. Code is available at https://github.com/pnu-cvsp/Enhancing-Stability-in-Training-Conditional-GAN-via-Selective-Data-Matching.

We propose a two-level dimmer based on binary distribution matching where a low-rate signal controls the output probability distribution of a high-rate bit sequence, which can be used in region-of-interest (RoI) signaling applications. To reduce the rate loss of the dimmer, we propose the extended multiset-partition distribution matching (EMPDM) algorithm with a novel binary-tree-structure implementation. In addition, we introduce 4p-EMPDM, a compact version of EMPDM, which has a typical composition (TC) and four leading composition pairs (CPs). The codebook of 4p-EMPDM includes only run-length-aware codewords, which reduces the maximum run-length of the transmitted bit sequence by 4.27 times. Hence, it guarantees flicker mitigation for visible-light RoI signaling systems at 6 kHz without using any run-length limited code (non-RLL). Using experimental data collected from the low-rate RoI signaling prototype, we introduced a threshold range where both intensity and area information of the received training symbols can be exploited to optimize the shaping ratios of the 4p-EMPDM dimmer. Because of the non-RLL feature, the proposed system can support soft-decision forward-error-correction (FEC) decoding to improve reliability. Our system outperforms related systems based on hybrid modulation schemes in terms of spectral efficiency, bit rate, and minimum required optical clock rate.

Multimodally distributed data is very common in remote sensing images, such as hyperspectral images (HSIs). It is important to capture the local manifold structure while preserving the global discriminant information in the multimodal data. In this article, an adaptive local discriminant analysis and distribution matching (ALDADM) method is designed for the domain adaptation (DA) in HSI classification. ALDADM uses adaptive local discriminant analysis to extract features that are discriminative and robust to multimodally distributed data. Meanwhile, considering the spectral properties of HSI, the domain shift is reduced by distribution matching and subspace alignment. During this learning process, ALDADM only selects density peak samples to reduce the influence of interfering samples. Four DA tasks are designing using University of Pavia, Center of Pavia, Yancheng, and Botswana HSI datasets. Compared with existing DA methods, the experimental results demonstrate that our ALDADM offers a superior performance.

In this paper, multi-dimensional distribution matching (MDDM) based on bit-level shaping is proposed to implement probabilistic shaping (PS) for high order modulation formats. MDDM is optimized across multiple dimensions including polarizations and different time slots to achieve better performance. The implementation complexity of PS can also be reduced by optimizing the multiple bit-level distribution matchers (DMs) for generating multi-dimensional symbols. To further reduce the complexity, degenerate MDDM is also proposed with almost the same performance as MDDM. In simulations of an additive white Gaussian noise (AWGN) channel, for probabilistically shaped 256-ary quadrature amplitude modulation (256QAM) at a shaping rate of 2.2 bits/amplitude with a block length of 80, four-dimensional distribution matching (4D-DM) and degenerate 4D-DM provide normalized generalized mutual information (NGMI) gains of 0.047 and 0.046 over constant composition distribution matching (CCDM), respectively. The performance of 4D-DM and degenerate 4D-DM is also evaluated by experiments. Compared to CCDM, degenerate 4D-DM achieves a NGMI gain of 0.055. [ABSTRACT FROM AUTHOR]

In this work, we implement a complete probabilistic amplitude shaping (PAS) architecture on a field-programmable gate array (FPGA) platform to study the interplay between probabilistic shaping (PS) and forward error correction (FEC). Due to the fully parallelized input–output interfaces based on look up table (LUT) and low computational complexity without high-precision multiplication, hierarchical distribution matching (HiDM) is chosen as the solution for real time probabilistic shaping. In terms of FEC, we select two kinds of the mainstream soft decision-forward error correction (SD-FEC) algorithms currently used in optical communication system, namely Open FEC (OFEC) and soft-decision quasi-cyclic low-density parity-check (SD-QC-LDPC) codes. Through FPGA experimental investigation, we studied the impact of probabilistic shaping on OFEC and LDPC, respectively, based on PS-16QAM under moderate shaping, and also the impact of probabilistic shaping on LDPC code based on PS-64QAM under weak/strong shaping. The FPGA experimental results show that if pre-FEC bit error rate (BER) is used as the predictor, moderate shaping induces no degradation on the OFEC performance, while strong shaping slightly degrades the error correction performance of LDPC. Nevertheless, there is no error floor when the output BER is around 10-15. However, if normalized generalized mutual information (NGMI) is selected as the predictor, the performance degradation of LDPC will become insignificant, which means pre-FEC BER may not a good predictor for LDPC in probabilistic shaping scenario. We also studied the impact of residual errors after FEC decoding on HiDM. The FPGA experimental results show that the increased BER after HiDM decoding is within 10 times compared to post-FEC BER. [ABSTRACT FROM AUTHOR]

Graphs with complete node attributes have been widely explored recently. While in practice, there is a graph where attributes of only partial nodes could be available and those of the others might be entirely missing. This attribute-missing graph is related to numerous real-world applications and there are limited studies investigating the corresponding learning problems. Existing graph learning methods including the popular GNN cannot provide satisfied learning performance since they are not specified for attribute-missing graphs. Thereby, designing a new GNN for these graphs is a burning issue to the graph learning community. In this article, we make a shared-latent space assumption on graphs and develop a novel distribution matching-based GNN called structure-attribute transformer (SAT) for attribute-missing graphs. SAT leverages structures and attributes in a decoupled scheme and achieves the joint distribution modeling of structures and attributes by distribution matching techniques. It could not only perform the link prediction task but also the newly introduced node attribute completion task. Furthermore, practical measures are introduced to quantify the performance of node attribute completion. Extensive experiments on seven real-world datasets indicate SAT shows better performance than other methods on both link prediction and node attribute completion tasks. [ABSTRACT FROM AUTHOR]

Transfer learning aims to learn an effective classifier for unlabeled target data by borrowing knowledge from well-labeled source data. However, most existing work has emphasized on learning domain invariant features to reduce the distribution discrepancy, which may suffer from the negative transfer problem caused by structure inconsistencies or distribution outliers. To address this challenge, in this paper, we propose a novel transfer learning approach, which seamlessly integrates domain invariant feature learning, discriminative structure preservation and sample reweighting into a unified learning model. Specifically, we attempt to learn domain invariant features by jointly adapting the marginal and conditional distributions. To transfer discriminative knowledge inferred from data, we enforce the structure consistency between the original feature space and the latent feature space. Furthermore, to enhance the robustness of our model, an efficient and more generalized sample reweighting strategy is developed to assign target predictions with different levels of confidence. The key advantage over previous methods is that our model can adaptively select pivot samples in target domain and retain the properties of discriminative structures underlying data domains, which enables coupled knowledge transfer during the learning process. Experimental results on several benchmark datasets have verified the superiority of the proposed method over other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

A general distribution matching architecture based on a multi-level structure is presented. It allows to generate arbitrary symbol distributions using simple binary distribution matchers. This is particularly advantageous for large symbol alphabets, where non-binary distribution matching algorithms tend to have high complexity. Some examples for possible implementations are provided, and a specific realization based on polar codes, which enables joint shaping and channel coding for reliable data transmission with higher-order modulation, is discussed and evaluated in detail. [ABSTRACT FROM AUTHOR]

Shaping codes are used to encode information for use on channels with cost constraints. Applications include data transmission with a power constraint and, more recently, data storage on flash memories with a constraint on memory cell wear. In the latter application, system requirements often impose a rate constraint. In this paper, we study rate-constrained fixed-to-variable length shaping codes for noiseless, memoryless costly channels and general i.i.d. sources. The analysis relies on the theory of word-valued sources. We establish a relationship between the code expansion factor – the ratio of the expected codeword length to the length of the input source word – and the minimum average symbol cost. We then determine the expansion factor that minimizes the average cost per source symbol (total cost), corresponding to a conventional optimal source code with cost. An equivalence is established between codes minimizing average symbol cost and codes minimizing total cost, and a separation theorem is proved, showing that optimal shaping can be achieved by a concatenation of optimal compression and optimal shaping for a uniform i.i.d. source. Shaping codes often incorporate, either explicitly or implicitly, some form of non-equiprobable signaling. We use our results to further explore the connections between shaping codes and codes that map a sequence of i.i.d. source symbols into an output sequence of symbols that are approximately independent and distributed according to a specified target distribution, such as distribution matching (DM) codes. Optimal DM codes are characterized in terms of a new performance measure - generalized expansion factor (GEF) - motivated by the costly channel perspective. The GEF is used to study DM codes that minimize informational divergence and normalized informational divergence. [ABSTRACT FROM AUTHOR]

In this article, we study amplitude shaping schemes for the probabilistic amplitude shaping (PAS) framework as well as algorithms for constant-composition distribution matching (CCDM). Huffman-coded sphere shaping (HCSS) is discussed in detail, which internally uses Huffman coding to determine the composition to be used and relies on conventional CCDM algorithms for mapping and demapping. Numerical simulations show that HCSS closes the performance gap between distribution matching schemes and sphere shaping techniques such as enumerative sphere shaping (ESS). HCSS is based on an architecture that is different from the trellis-based setup of ESS. It allows to tailor the used HCSS compositions to the transmission channel and to take into account complexity constraints. We further discuss in detail multiset ranking (MR) and subset ranking (SR) as alternatives to arithmetic-coding (AC) CCDM. The advantage of MR over AC is that it requires less sequential operations for mapping. SR operates on binary alphabets only, which can introduce some additional rate loss when a nonbinary-to-binary transformation is required. However, the binomial coefficients required for SR can be precomputed and stored in a lookup table (LUT). We perform an analysis of rate loss and decoding performance for the proposed techniques and compare them to other prominent amplitude shaping schemes. For medium to long block lengths, MR-HCSS and SR-HCSS are shown to have similar performance to ESS. SR-HCSS and uniform 64QAM are compared in additive white Gaussian noise simulations and shaping gains of 0.5 dB and 1 dB are demonstrated with 1 kbit and 100 kbit LUT size, respectively. [ABSTRACT FROM AUTHOR]

The implementation difficulties of combining distribution matching (DM) and dematching (invDM) for probabilistic shaping (PS) with soft-decision forward error correction (FEC) coding can be relaxed by reverse concatenation, for which the FEC coding and decoding lies inside the shaping algorithms. PS can seemingly achieve performance close to the Shannon limit, although there are practical implementation challenges that need to be carefully addressed. We propose a hierarchical DM (HiDM) scheme, having fully parallelized input–output interfaces and a pipelined architecture that can efficiently perform the DM/invDM without the complex operations of previously proposed methods such as constant composition DM. Furthermore, HiDM can operate at a significantly larger post-FEC bit error rate (BER) for the same post-invDM BER performance, which facilitates simulations. These benefits come at the cost of a slightly larger rate loss and required signal-to-noise ratio at a given post-FEC BER. [ABSTRACT FROM AUTHOR]

Distribution matching is a fixed-length invertible mapping from a uniformly distributed bit sequence to shaped amplitudes and plays an important role in the probabilistic amplitude shaping framework. With conventional constant-composition distribution matching (CCDM), all output sequences have identical composition. In this paper, we propose multiset-partition distribution matching (MPDM), where the composition is constant over all output sequences. When considering the desired distribution as a multiset, MPDM corresponds to partitioning this multiset into equal-sized subsets. We show that MPDM allows addressing more output sequences and, thus, has a lower rate loss than CCDM in all nontrivial cases. By imposing some constraints on the partitioning, a constructive MPDM algorithm is proposed which comprises two parts. A variable-length prefix of the binary data word determines the composition to be used, and the remainder of the input word is mapped with a conventional CCDM algorithm, such as arithmetic coding, according to the chosen composition. Simulations of 64-ary quadrature amplitude modulation over the additive white Gaussian noise channel demonstrate that the block-length saving of MPDM over CCDM for a fixed gap to capacity is approximately a factor of 2.5–5 at medium to high signal-to-noise ratios. [ABSTRACT FROM AUTHOR]

A routine to parallelize and increase the throughput of fixed length distribution matching with non-binary output alphabet is presented. The routine performs parallel distribution matchings with binary output alphabets, and then maps the binary output streams to symbols from the primary, non-binary alphabet. Fixed length distribution matching is essential for probabilistically shaped coded modulation (PSCM), such as the probabilistic amplitude shaping (PAS). In the PAS setup, the novel bit-level distribution matcher achieves similar rate-versus-SNR performance as the symbol-level constant composition distribution matcher, thus providing a high throughput solution for PSCM. [ABSTRACT FROM PUBLISHER]

Many brain disorders and diseases exhibit heterogeneous symptoms and imaging characteristics. This heterogeneity is typically not captured by commonly adopted neuroimaging analyses that seek only a main imaging pattern when two groups need to be differentiated (e.g., patients and controls, or clinical progressors and non-progressors). We propose a novel probabilistic clustering approach, CHIMERA, modeling the pathological process by a combination of multiple regularized transformations from normal/control population to the patient population, thereby seeking to identify multiple imaging patterns that relate to disease effects and to better characterize disease heterogeneity. In our framework, normal and patient populations are considered as point distributions that are matched by a variant of the coherent point drift algorithm. We explain how the posterior probabilities produced during the MAP optimization of CHIMERA can be used for clustering the patients into groups and identifying disease subtypes. CHIMERA was first validated on a synthetic dataset and then on a clinical dataset mixing 317 control subjects and patients suffering from Alzheimer's Disease (AD) and Parkison's Disease (PD). CHIMERA produced better clustering results compared to two standard clustering approaches. We further analyzed 390 T1 MRI scans from Alzheimer's patients. We discovered two main and reproducible AD subtypes displaying significant differences in cognitive performance. [ABSTRACT FROM PUBLISHER]

Distribution matching transforms independent and Bernoulli(1/2) distributed input bits into a sequence of output symbols with a desired distribution. Fixed-to-fixed length, invertible, and low complexity encoders and decoders based on constant composition and arithmetic coding are presented. The encoder achieves the maximum rate, namely, the entropy of the desired distribution, asymptotically in the blocklength. Furthermore, the normalized divergence of the encoder output and the desired distribution goes to zero in the blocklength. [ABSTRACT FROM PUBLISHER]