Search

Traditional methods used in causal mediation analysis with continuous treatment often focus on estimating average causal effects, limiting their applicability in precision medicine. Machine learning techniques have emerged as a powerful approach for precisely estimating individualized causal effects. This paper proposes a novel method called CGAN-ICMA-CT that leverages Conditional Generative Adversarial Networks (CGANs) to infer individualized causal effects with continuous treatment. We thoroughly investigate the convergence properties of CGAN-ICMA-CT and show that the estimated distribution of our inferential conditional generator converges to the true conditional distribution under mild conditions. We conduct numerical experiments to validate the effectiveness of CGAN-ICMA-CT and compare it with four commonly used methods: linear regression, support vector machine regression, decision tree, and random forest regression. The results demonstrate that CGAN-ICMA-CT outperforms these methods regarding accuracy and precision. Furthermore, we apply the CGAN-ICMA-CT model to the real-world Job Corps dataset, showcasing its practical utility. By utilizing CGAN-ICMA-CT, we estimate the individualized causal effects of the Job Corps program on the number of arrests, providing insights into both direct effects and effects mediated through intermediate variables. Our findings confirm the potential of CGAN-ICMA-CT in advancing individualized causal mediation analysis with continuous treatment in precision medicine settings. [ABSTRACT FROM AUTHOR]

This paper introduces a novel approach to reducing statistical errors in generative models, with a specific focus on generative adversarial networks (GANs). Inspired by the error analysis of GANs, we find that statistical errors mainly arise from random sampling, leading to significant uncertainties in GANs. To address this issue, we propose a selective sampling mechanism called space‐filling sampling. Our method aims to increase the sampling probability in areas with insufficient data, thereby improving the learning performance of the generator. Theoretical analysis confirms the effectiveness of our approach in reducing statistical errors and accelerating convergence in GANs. This research represents a pioneering effort in targeting the reduction of statistical errors in GANs, and it demonstrates the potential for enhancing the training of other generative models. [ABSTRACT FROM AUTHOR]

Supervised classification algorithms usually require a large quantity of well-labeled samples for training to achieve satisfied performance. Nevertheless, it is prohibitively difficult to create such datasets with complete annotation. Unsupervised domain adaptation is able to deal with this problem via transferring knowledge from a relevant dataset with rich labels to an unlabeled target dataset. In this paper, a novel unsupervised domain adaptation method named feature transfer learning based on elastic embedding (EEFTL) is presented for image classification. Rather than make a rigid embedding that the binary label matrix is exactly equal to a linear function, EEFTL adopts an elastic embedding induced by the prediction label matrix. Specifically, EEFTL introduces a flexible regression residue term to model the mismatch between the embedded features of samples and the prediction labels. In addition, EEFTL integrates a label fitness term to effectively utilize the label information from the source samples, a distribution matching term to reduce the distances between domains in both the marginal and conditional distributions, and a manifold regularization term to preserve the sample-wise structure information under the elastic embedding. Extensive experiments are carried out on multiple benchmark datasets, and the results prove the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Most domain adaptive methods enhance the classification performance of target domain via overcoming the distribution difference between source and target domains while ignoring the nature of classifier, i.e., the data being correctly classified by classifier is due to the high separability of data. Therefore, inspired by this, we propose a simple yet effective domain adaptive method in accordance with the property of classifier, namely cross-domain Fisher discrimination criterion (CFDC). CDFC is intended to upgrade the inter-class discrimination and intra-class compactness of samples for the scenario where the target domain has few labeled samples. Specifically, we reconstruct the intra- and inter- class scatter matrices of Fisher's criterion from a geometrically intuitive perspective, enabling it to extract highly discriminative features from cross-domain samples. The reconstructed between-class scatter matrix aims to maximize the class separation of target samples, while the redefined within-class scatter matrix seeks to have samples from different domains densely clustered around the class center of target samples. Six widely used image datasets are involved to verify the effectiveness and efficiency of CFDC. Experimental results demonstrate that the performance of CFDC is significantly superior to similar methods and conventional unsupervised domain adaptation methods, comparable to non-deep semi-supervised domain adaptation methods, but with a significantly lower time consumption. Even compared with state-of-the-art deep semi-supervised domain adaptation methods, CFDC exhibits performance advantages on certain datasets. [ABSTRACT FROM AUTHOR]

Conditional generative adversarial networks (cGANs) have demonstrated remarkable success due to their class-wise controllability and superior quality for complex generation tasks. Typical cGANs solve the joint distribution matching problem by decomposing two easier sub-problems: marginal matching and conditional matching. In this paper, we proposes a simple but effective training methodology, selective focusing learning, which enforces the discriminator and generator to learn easy samples of each class rapidly while maintaining diversity. Our key idea is to selectively apply conditional and joint matching for the data in each mini-batch.Specifically, we first select the samples with the highest scores when sorted using the conditional term of the discriminator outputs (real and generated samples). Then we optimize the model using the selected samples with only conditional matching and the other samples with joint matching. From our toy experiments, we found that it is the best to apply only conditional matching to certain samples due to the content-aware optimization of the discriminator. We conducted experiments on ImageNet ( $64 \times 64$ and $128 \times 128$ ), CIFAR-10, CIFAR-100 datasets, and Mixture of Gaussian, noisy label settings to demonstrate that the proposed method can substantially (up to 35.18% in terms of FID) improve all indicators with 10 independent trials. Code is available at https://github.com/pnu-cvsp/Enhancing-Stability-in-Training-Conditional-GAN-via-Selective-Data-Matching.

We propose a deep generative approach to sampling from a conditional distribution based on a unified formulation of conditional distribution and generalized nonparametric regression function using the noise-outsourcing lemma. The proposed approach aims at learning a conditional generator, so that a random sample from the target conditional distribution can be obtained by transforming a sample drawn from a reference distribution. The conditional generator is estimated nonparametrically with neural networks by matching appropriate joint distributions using the Kullback-Liebler divergence. An appealing aspect of our method is that it allows either of or both the predictor and the response to be high-dimensional and can handle both continuous and discrete type predictors and responses. We show that the proposed method is consistent in the sense that the conditional generator converges in distribution to the underlying conditional distribution under mild conditions. Our numerical experiments with simulated and benchmark image data validate the proposed method and demonstrate that it outperforms several existing conditional density estimation methods. for this article are available online. [ABSTRACT FROM AUTHOR]

We propose a two-level dimmer based on binary distribution matching where a low-rate signal controls the output probability distribution of a high-rate bit sequence, which can be used in region-of-interest (RoI) signaling applications. To reduce the rate loss of the dimmer, we propose the extended multiset-partition distribution matching (EMPDM) algorithm with a novel binary-tree-structure implementation. In addition, we introduce 4p-EMPDM, a compact version of EMPDM, which has a typical composition (TC) and four leading composition pairs (CPs). The codebook of 4p-EMPDM includes only run-length-aware codewords, which reduces the maximum run-length of the transmitted bit sequence by 4.27 times. Hence, it guarantees flicker mitigation for visible-light RoI signaling systems at 6 kHz without using any run-length limited code (non-RLL). Using experimental data collected from the low-rate RoI signaling prototype, we introduced a threshold range where both intensity and area information of the received training symbols can be exploited to optimize the shaping ratios of the 4p-EMPDM dimmer. Because of the non-RLL feature, the proposed system can support soft-decision forward-error-correction (FEC) decoding to improve reliability. Our system outperforms related systems based on hybrid modulation schemes in terms of spectral efficiency, bit rate, and minimum required optical clock rate.

Abstract Cross-domain classification refers to completing the corresponding classification task in a target domain which lacks label information, by exploring useful knowledge in a related source domain but with different data distribution. Domain adaptation can deal with such cross-domain classification, by reducing divergence of domains and transferring the relevant knowledge from the source to the target. To mine the discriminant information of the source domain samples and the geometric structure information of domains, and thus improve domain adaptation performance, this paper proposes a novel method involving distribution matching and structure preservation for domain adaptation (DMSP). First, it aligns the subspaces of the source domain and target domain on the Grassmann manifold; and learns the non-distorted embedded feature representations of the two domains. Second, in this embedded feature space, the empirical structure risk minimization method with distribution adaptation regularization and intra-domain graph regularization is used to learn an adaptive classifier, further adapting the source and target domains. Finally, we perform extensive experiments on widely used cross-domain classification datasets to validate the superiority of DMSP. The average classification accuracy of DMSP on these datasets is the highest compared with several state-of-the-art domain adaptation methods.

Cross-domain classification refers to completing the corresponding classification task in a target domain which lacks label information, by exploring useful knowledge in a related source domain but with different data distribution. Domain adaptation can deal with such cross-domain classification, by reducing divergence of domains and transferring the relevant knowledge from the source to the target. To mine the discriminant information of the source domain samples and the geometric structure information of domains, and thus improve domain adaptation performance, this paper proposes a novel method involving distribution matching and structure preservation for domain adaptation (DMSP). First, it aligns the subspaces of the source domain and target domain on the Grassmann manifold; and learns the non-distorted embedded feature representations of the two domains. Second, in this embedded feature space, the empirical structure risk minimization method with distribution adaptation regularization and intra-domain graph regularization is used to learn an adaptive classifier, further adapting the source and target domains. Finally, we perform extensive experiments on widely used cross-domain classification datasets to validate the superiority of DMSP. The average classification accuracy of DMSP on these datasets is the highest compared with several state-of-the-art domain adaptation methods. [ABSTRACT FROM AUTHOR]

Multimodally distributed data is very common in remote sensing images, such as hyperspectral images (HSIs). It is important to capture the local manifold structure while preserving the global discriminant information in the multimodal data. In this article, an adaptive local discriminant analysis and distribution matching (ALDADM) method is designed for the domain adaptation (DA) in HSI classification. ALDADM uses adaptive local discriminant analysis to extract features that are discriminative and robust to multimodally distributed data. Meanwhile, considering the spectral properties of HSI, the domain shift is reduced by distribution matching and subspace alignment. During this learning process, ALDADM only selects density peak samples to reduce the influence of interfering samples. Four DA tasks are designing using University of Pavia, Center of Pavia, Yancheng, and Botswana HSI datasets. Compared with existing DA methods, the experimental results demonstrate that our ALDADM offers a superior performance.

The increasing prevalence of multi-site diffusion-weighted magnetic resonance imaging (dMRI) studies potentially offers enhanced statistical power for investigating brain structure. However, these studies face challenges due to variations in scanner hardware and acquisition protocols. While several methods exist for dMRI data harmonization, few specifically address structural brain connectivity. We introduce a new distribution-matching approach to harmonizing structural brain connectivity across different sites and scanners. We evaluate our method using structural brain connectivity data from two distinct datasets of OASIS-3 and ADNI-2, comparing its performance to the widely used ComBat method. Our approach is meant to align the statistical properties of connectivity data from these two datasets. We examine the impact of harmonization on the correlation of brain connectivity with the Mini-Mental State Examination score and age. Our results demonstrate that our distribution-matching technique more effectively harmonizes structural brain connectivity, often producing stronger and more significant correlations compared to ComBat. Qualitative assessments illustrate the desired distributional alignment of ADNI-2 with OASIS-3, while quantitative evaluations confirm robust performance. This work contributes to the growing field of dMRI harmonization, potentially improving the reliability and comparability of structural connectivity studies that combine data from different sources in neuroscientific and clinical research., Competing Interests: Declaration of Competing Interests B. Fischl is an advisor to DeepHealth, a company whose medical pursuits focus on medical imaging and measurement technologies. His interests were reviewed and are managed by Massachusetts General Hospital and Mass General Brigham in accordance with their conflict-of-interest policies. The other authors have nothing to disclose.

Heterogeneous domain adaptation (HDA) aims to adapt a trained model on a source domain with different input feature space to an unlabeled target domain. In fact, HDA is a challenging issue, since there exists feature and distribution discrepancies across domains. In this paper, we propose a novel approach named as heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection (SDA-PPLS). SDA-PPLS learns two projection matrices for source and target domains to map them into a latent subspace to have a shared feature space. Moreover, to mitigate the distribution gap, SDA-PPLS aligns both first-order and second-order statistical information, simultaneously, to improve the target classification model performance. In addition, to discriminate instances into distinct classes, SDA-PPLS aligns the class conditional distributions by pseudo label refinement of target domain data. Finally, to prevent the propagation of inaccurate pseudo labels to the next iteration, a progressive technique is proposed to select instances with higher probability. Experimental results on several real-word datasets on image to image, text to text and text to image tasks with different feature representations, demonstrate that the proposed method outperforms other state-of-the-art HDA methods. [ABSTRACT FROM AUTHOR]

In this paper, multi-dimensional distribution matching (MDDM) based on bit-level shaping is proposed to implement probabilistic shaping (PS) for high order modulation formats. MDDM is optimized across multiple dimensions including polarizations and different time slots to achieve better performance. The implementation complexity of PS can also be reduced by optimizing the multiple bit-level distribution matchers (DMs) for generating multi-dimensional symbols. To further reduce the complexity, degenerate MDDM is also proposed with almost the same performance as MDDM. In simulations of an additive white Gaussian noise (AWGN) channel, for probabilistically shaped 256-ary quadrature amplitude modulation (256QAM) at a shaping rate of 2.2 bits/amplitude with a block length of 80, four-dimensional distribution matching (4D-DM) and degenerate 4D-DM provide normalized generalized mutual information (NGMI) gains of 0.047 and 0.046 over constant composition distribution matching (CCDM), respectively. The performance of 4D-DM and degenerate 4D-DM is also evaluated by experiments. Compared to CCDM, degenerate 4D-DM achieves a NGMI gain of 0.055. [ABSTRACT FROM AUTHOR]

In this work, we implement a complete probabilistic amplitude shaping (PAS) architecture on a field-programmable gate array (FPGA) platform to study the interplay between probabilistic shaping (PS) and forward error correction (FEC). Due to the fully parallelized input–output interfaces based on look up table (LUT) and low computational complexity without high-precision multiplication, hierarchical distribution matching (HiDM) is chosen as the solution for real time probabilistic shaping. In terms of FEC, we select two kinds of the mainstream soft decision-forward error correction (SD-FEC) algorithms currently used in optical communication system, namely Open FEC (OFEC) and soft-decision quasi-cyclic low-density parity-check (SD-QC-LDPC) codes. Through FPGA experimental investigation, we studied the impact of probabilistic shaping on OFEC and LDPC, respectively, based on PS-16QAM under moderate shaping, and also the impact of probabilistic shaping on LDPC code based on PS-64QAM under weak/strong shaping. The FPGA experimental results show that if pre-FEC bit error rate (BER) is used as the predictor, moderate shaping induces no degradation on the OFEC performance, while strong shaping slightly degrades the error correction performance of LDPC. Nevertheless, there is no error floor when the output BER is around 10-15. However, if normalized generalized mutual information (NGMI) is selected as the predictor, the performance degradation of LDPC will become insignificant, which means pre-FEC BER may not a good predictor for LDPC in probabilistic shaping scenario. We also studied the impact of residual errors after FEC decoding on HiDM. The FPGA experimental results show that the increased BER after HiDM decoding is within 10 times compared to post-FEC BER. [ABSTRACT FROM AUTHOR]

Graphs with complete node attributes have been widely explored recently. While in practice, there is a graph where attributes of only partial nodes could be available and those of the others might be entirely missing. This attribute-missing graph is related to numerous real-world applications and there are limited studies investigating the corresponding learning problems. Existing graph learning methods including the popular GNN cannot provide satisfied learning performance since they are not specified for attribute-missing graphs. Thereby, designing a new GNN for these graphs is a burning issue to the graph learning community. In this article, we make a shared-latent space assumption on graphs and develop a novel distribution matching-based GNN called structure-attribute transformer (SAT) for attribute-missing graphs. SAT leverages structures and attributes in a decoupled scheme and achieves the joint distribution modeling of structures and attributes by distribution matching techniques. It could not only perform the link prediction task but also the newly introduced node attribute completion task. Furthermore, practical measures are introduced to quantify the performance of node attribute completion. Extensive experiments on seven real-world datasets indicate SAT shows better performance than other methods on both link prediction and node attribute completion tasks. [ABSTRACT FROM AUTHOR]

This paper investigates how data requirements often encountered in archival accounting research can produce a data-restricted sample that is a non-random selection of observations from the reference sample to which the researcher wishes to generalize results. We illustrate the effects of non-random sampling on results of association tests in a setting with data on one variable of interest for all observations and frequently-missing data on another variable of interest. We develop and validate a resampling approach that uses only observations from the data-restricted sample to construct distribution-matched samples that approximate randomly-drawn samples from the reference sample. Our simulation tests provide evidence that distribution-matched samples yield generalizable results. We demonstrate the effects of non-random sampling in tests of the association between realized returns and five implied cost of equity metrics. In this setting, the reference sample has full information on realized returns, while on average only 16% of reference sample observations have data on cost of equity metrics. Consistent with prior research (e.g., Easton and Monahan The Accounting Review 80, 501–538, 2005), analysis using the unadjusted (non-random) cost of equity sample reveals weak or negative associations between realized returns and cost of equity metrics. In contrast, using distribution-matched samples, we find reliable evidence of the theoretically-predicted positive association. We also conceptually and empirically compare distribution-matching with multiple imputation and selection models, two other approaches to dealing with non-random samples. [ABSTRACT FROM AUTHOR]

For the machine learning-based prediction of the conversion from mild cognitive impairment to Alzheimer's disease, the collection of sufficient data to train a model is required, which involves a lot of time and expense. When data is not enough, combining public and in-house data may be appropriate by applying domain adaptation that alleviates inter-site heterogeneity. Existing methods simultaneously transform in-house and public data to represent them into a common feature space, and then train a classifier using labels in public data. However, this procedure causes the time- and cost-consuming re-training of classifier whenever in-house data changes, and also inheres the risk of information loss in public data. Motivated by this, we propose a method that only transforms in-house data while preserving public data, namely one-way domain adaptation. The proposed method represents in-house data similar with public data by matching the data distribution and the connectivity between brain regions with mean vectors and covariance matrices, respectively. Then, the pre-trained classifier in public data is applied to predict AD conversion for in-house data. The experiments, which use the Australian Imaging Biomarkers and Lifestyle Study of Aging and the Open Access Series of Imaging Studies as the in-house data and the Alzheimer's Disease Neuroimaging Initiative as the public data, show the effectiveness and efficiency of the proposed method, improving prediction performance about 34.8% on average without labels in the in-house datasets. [ABSTRACT FROM AUTHOR]

Transfer learning aims to learn an effective classifier for unlabeled target data by borrowing knowledge from well-labeled source data. However, most existing work has emphasized on learning domain invariant features to reduce the distribution discrepancy, which may suffer from the negative transfer problem caused by structure inconsistencies or distribution outliers. To address this challenge, in this paper, we propose a novel transfer learning approach, which seamlessly integrates domain invariant feature learning, discriminative structure preservation and sample reweighting into a unified learning model. Specifically, we attempt to learn domain invariant features by jointly adapting the marginal and conditional distributions. To transfer discriminative knowledge inferred from data, we enforce the structure consistency between the original feature space and the latent feature space. Furthermore, to enhance the robustness of our model, an efficient and more generalized sample reweighting strategy is developed to assign target predictions with different levels of confidence. The key advantage over previous methods is that our model can adaptively select pivot samples in target domain and retain the properties of discriminative structures underlying data domains, which enables coupled knowledge transfer during the learning process. Experimental results on several benchmark datasets have verified the superiority of the proposed method over other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

A general distribution matching architecture based on a multi-level structure is presented. It allows to generate arbitrary symbol distributions using simple binary distribution matchers. This is particularly advantageous for large symbol alphabets, where non-binary distribution matching algorithms tend to have high complexity. Some examples for possible implementations are provided, and a specific realization based on polar codes, which enables joint shaping and channel coding for reliable data transmission with higher-order modulation, is discussed and evaluated in detail. [ABSTRACT FROM AUTHOR]

Shaping codes are used to encode information for use on channels with cost constraints. Applications include data transmission with a power constraint and, more recently, data storage on flash memories with a constraint on memory cell wear. In the latter application, system requirements often impose a rate constraint. In this paper, we study rate-constrained fixed-to-variable length shaping codes for noiseless, memoryless costly channels and general i.i.d. sources. The analysis relies on the theory of word-valued sources. We establish a relationship between the code expansion factor – the ratio of the expected codeword length to the length of the input source word – and the minimum average symbol cost. We then determine the expansion factor that minimizes the average cost per source symbol (total cost), corresponding to a conventional optimal source code with cost. An equivalence is established between codes minimizing average symbol cost and codes minimizing total cost, and a separation theorem is proved, showing that optimal shaping can be achieved by a concatenation of optimal compression and optimal shaping for a uniform i.i.d. source. Shaping codes often incorporate, either explicitly or implicitly, some form of non-equiprobable signaling. We use our results to further explore the connections between shaping codes and codes that map a sequence of i.i.d. source symbols into an output sequence of symbols that are approximately independent and distributed according to a specified target distribution, such as distribution matching (DM) codes. Optimal DM codes are characterized in terms of a new performance measure - generalized expansion factor (GEF) - motivated by the costly channel perspective. The GEF is used to study DM codes that minimize informational divergence and normalized informational divergence. [ABSTRACT FROM AUTHOR]

In this article, we study amplitude shaping schemes for the probabilistic amplitude shaping (PAS) framework as well as algorithms for constant-composition distribution matching (CCDM). Huffman-coded sphere shaping (HCSS) is discussed in detail, which internally uses Huffman coding to determine the composition to be used and relies on conventional CCDM algorithms for mapping and demapping. Numerical simulations show that HCSS closes the performance gap between distribution matching schemes and sphere shaping techniques such as enumerative sphere shaping (ESS). HCSS is based on an architecture that is different from the trellis-based setup of ESS. It allows to tailor the used HCSS compositions to the transmission channel and to take into account complexity constraints. We further discuss in detail multiset ranking (MR) and subset ranking (SR) as alternatives to arithmetic-coding (AC) CCDM. The advantage of MR over AC is that it requires less sequential operations for mapping. SR operates on binary alphabets only, which can introduce some additional rate loss when a nonbinary-to-binary transformation is required. However, the binomial coefficients required for SR can be precomputed and stored in a lookup table (LUT). We perform an analysis of rate loss and decoding performance for the proposed techniques and compare them to other prominent amplitude shaping schemes. For medium to long block lengths, MR-HCSS and SR-HCSS are shown to have similar performance to ESS. SR-HCSS and uniform 64QAM are compared in additive white Gaussian noise simulations and shaping gains of 0.5 dB and 1 dB are demonstrated with 1 kbit and 100 kbit LUT size, respectively. [ABSTRACT FROM AUTHOR]

Image processing has attracted increasing attention in recent researches to solve domain shift problem where machine learning algorithms are applied to sets of unseen images. Domain shift problem occurs when the training (source domain) and test (target domain) sets are collected in different environmental conditions but in related domains. In this way, the adaptation across data distributions of the source and target datasets are suggested as domain adaptation framework to overcome the performance degradation. In this paper, a novel domain adaptation method referred as joint local and statistical discriminant learning via feature alignment (LSA), is proposed to find a cross-domain subspace by matching cross-domain distribution shift and adapting the class structures of the local and statistical distributions across the source and target domains, during the dimensionality reduction. Specifically, LSA projects samples into an embedded subspace in which the distances across the samples from same class are minimized and the distances across samples from different classes are maximized, at each local and statistical area, during alignment of marginal and conditional distributions. Furthermore, the class densities of samples based on manifold structure in different classes are preserved to provide more separability across various classes. To evaluate the proposed method, comprehensive experiments have been conducted on benchmark cross-domain object and digit recognition datasets. Experimental results have verified the superiority of LSA with a large margin in average classification accuracy against several state-of-the-art distribution matching and discriminant learning methods of domain adaptation. Moreover, the results have demonstrated the effectiveness of our proposed representation learning. Our source code is available at https://github.com/jtahmores/LSA. [ABSTRACT FROM AUTHOR]

The implementation difficulties of combining distribution matching (DM) and dematching (invDM) for probabilistic shaping (PS) with soft-decision forward error correction (FEC) coding can be relaxed by reverse concatenation, for which the FEC coding and decoding lies inside the shaping algorithms. PS can seemingly achieve performance close to the Shannon limit, although there are practical implementation challenges that need to be carefully addressed. We propose a hierarchical DM (HiDM) scheme, having fully parallelized input–output interfaces and a pipelined architecture that can efficiently perform the DM/invDM without the complex operations of previously proposed methods such as constant composition DM. Furthermore, HiDM can operate at a significantly larger post-FEC bit error rate (BER) for the same post-invDM BER performance, which facilitates simulations. These benefits come at the cost of a slightly larger rate loss and required signal-to-noise ratio at a given post-FEC BER. [ABSTRACT FROM AUTHOR]

Distribution matching is a fixed-length invertible mapping from a uniformly distributed bit sequence to shaped amplitudes and plays an important role in the probabilistic amplitude shaping framework. With conventional constant-composition distribution matching (CCDM), all output sequences have identical composition. In this paper, we propose multiset-partition distribution matching (MPDM), where the composition is constant over all output sequences. When considering the desired distribution as a multiset, MPDM corresponds to partitioning this multiset into equal-sized subsets. We show that MPDM allows addressing more output sequences and, thus, has a lower rate loss than CCDM in all nontrivial cases. By imposing some constraints on the partitioning, a constructive MPDM algorithm is proposed which comprises two parts. A variable-length prefix of the binary data word determines the composition to be used, and the remainder of the input word is mapped with a conventional CCDM algorithm, such as arithmetic coding, according to the chosen composition. Simulations of 64-ary quadrature amplitude modulation over the additive white Gaussian noise channel demonstrate that the block-length saving of MPDM over CCDM for a fixed gap to capacity is approximately a factor of 2.5–5 at medium to high signal-to-noise ratios. [ABSTRACT FROM AUTHOR]

• Intra -period discrepancy between past and future unseen degradation periods is considered. • Compound domain shifts (CDS) for cross-domain machinery degradation prognostics is formulated. • The cross-domain prediction performances are greatly improved after resolving CDS problem. • Public and real-measured tests prove the effectiveness of the proposed prediction method. Nowadays, data-driven based decision-making mode significantly promotes machinery prognostics and health management (PHM), but are also profoundly affected by domain shift problems. As a promising transfer learning methodology, domain adaptation has matured in machinery fault classifications under variable operating conditions and achieved tremendous success in aligning data distribution discrepancies among multiple domains, called inter -domain shifts in this study. However the monitoring signal presents nonlinear and non-stationary characteristics when the machine is degraded, its period-level and chronological discrepancy within the given set of monitoring time series is often ignored. This intra discrepancy is not conducive to maintaining the time series consistency and may fail to build an accurate prediction model. Therefore, this study, for the first time, formulas this kind of shift and comprehensively resolves the compound shifts originating from inter and intra domains. Firstly, the domain-invariant degradation indicator is constructed through the designed double adversarial learning based multi-source domain adaptation module. Then the obtained indicators are segmented into distinct degradation periods according to outlier detections for subsequent intra -domain level alignments, quantifying as the regularization term for matching the discrepancies between degradation periods. Finally, real-measured run-to-failed experiments are applied to test the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we provide a systematic comparison of distribution matching (DM) and sphere shaping (SpSh) algorithms for short blocklength probabilistic amplitude shaping. For asymptotically large blocklengths, constant composition distribution matching (CCDM) is known to generate the target capacity-achieving distribution. However, as the blocklength decreases, the resulting rate loss diminishes the efficiency of CCDM. We claim that for such short blocklengths over the additive white Gaussian noise (AWGN) channel, the objective of shaping should be reformulated as obtaining the most energy-efficient signal space for a given rate (rather than matching distributions). In light of this interpretation, multiset-partition DM (MPDM) and SpSh are reviewed as energy-efficient shaping techniques. Numerical results show that both have smaller rate losses than CCDM. SpSh—whose sole objective is to maximize the energy efficiency—is shown to have the minimum rate loss amongst all, which is particularly apparent for ultra short blocklengths. We provide simulation results of the end-to-end decoding performance showing that up to 1 dB improvement in power efficiency over uniform signaling can be obtained with MPDM and SpSh at blocklengths around 200. Finally, we present a discussion on the complexity of these algorithms from the perspectives of latency, storage and computations.

Transfer learning is a problem defined over two domains. These two domains share the same feature space and class label space, but have significantly different distributions. One domain has sufficient labels, named as source domain, and the other domain has few labels, named as target domain. The problem is to learn an effective classifier for the target domain. In this paper, we propose a novel transfer learning method for this problem by learning a partially shared classifier for the target domain, and weighting the source domain data points. We learn some shared subspaces for both the data points of the two domains, and a shared classifier in the shared subspaces. We hope that in the shared subspaces, the distributions of two domain can match each other well, and to match the distributions, we weight the source domain data points with different weighting factors. Moreover, we adapt the shared classifier to each domain by learning different adaptation functions. To learn the subspace transformation matrices, the classifier parameters, and the adaptation parameters, we build an objective function with weighted classification errors, parameter regularization, local reconstruction regularization, and distribution matching. This objective function is minimized by an iterative algorithm. Experiments show its effectiveness over benchmark data sets, including travel destination review data set, face expression data set, spam email data set. [ABSTRACT FROM AUTHOR]

Active learning aims to interactively select the most informative data for accelerating the learning procedure. In this paper, we propose a novel and principled deep active learning approach under label shift —the labeled dataset has a different class distribution with respect to the unlabeled data pool. By formulating the querying procedure as a distribution matching problem, we derive the generalization guarantee under label shift. Moreover, the theoretical results inspire a unified practical framework for deep batch active learning: in the training stage, we jointly correct the label shift and align the semantic conditional distributions; as for the querying stage, label-shift-conditioned hybrid querying (LSCHQ) strategy is proposed to balance uncertainty and diversity under label shift. Empirical results show that our proposed method is highly effective and outperforms other baseline algorithms over a variety of benchmarks and a real-world dataset. • Theoretical analysis for deep active learning under label shift. • Label shift correction and conditional distribution alignment during training. • Querying strategy considers uncertainty, diversity and the affect of label shift. [ABSTRACT FROM AUTHOR]

Purpose: This study investigates an efficient (nearly real-time) two-stage spine labeling algorithm that removes the need for an external training while being applicable to different types of MRI data and acquisition protocols. Methods: Based solely on the image being labeled (i.e., we do not use training data), the first stage aims at detecting potential vertebra candidates following the optimization of a functional containing two terms: (i) a distribution-matching term that encodes contextual information about the vertebrae via a density model learned from a very simple user input, which amounts to a point (mouse click) on a predefined vertebra; and (ii) a regularization constraint, which penalizes isolated candidates in the solution. The second stage removes false positives and identifies all vertebrae and discs by optimizing a geometric constraint, which embeds generic anatomical information on the interconnections between neighboring structures. Based on generic knowledge, our geometric constraint does not require external training. Results: We performed quantitative evaluations of the algorithm over a data set of 90 mid-sagittal MRI images of the lumbar spine acquired from 45 different subjects. To assess the flexibility of the algorithm, we used both T1- and T2-weighted images for each subject. A total of 990 structures were automatically detected/labeled and compared to ground-truth annotations by an expert. On the T2-weighted data, we obtained an accuracy of 91.6% for the vertebrae and 89.2% for the discs. On the T1-weighted data, we obtained an accuracy of 90.7% for the vertebrae and 88.1% for the discs. Conclusion: Our algorithm removes the need for external training while being applicable to different types of MRI data and acquisition protocols. Based on the current testing data, a subject-specific model density and generic anatomical information, our method can achieve competitive performances when applied to T1- and T2-weighted MRI images. [ABSTRACT FROM AUTHOR]

A routine to parallelize and increase the throughput of fixed length distribution matching with non-binary output alphabet is presented. The routine performs parallel distribution matchings with binary output alphabets, and then maps the binary output streams to symbols from the primary, non-binary alphabet. Fixed length distribution matching is essential for probabilistically shaped coded modulation (PSCM), such as the probabilistic amplitude shaping (PAS). In the PAS setup, the novel bit-level distribution matcher achieves similar rate-versus-SNR performance as the symbol-level constant composition distribution matcher, thus providing a high throughput solution for PSCM. [ABSTRACT FROM PUBLISHER]

Multi-label learning is an effective framework for learning with objects that have multiple semantic labels, and has been successfully applied into many real-world tasks. In contrast with traditional single-label learning, the cost of labeling a multi-label example is rather high, thus it becomes an important task to train an effectivemulti-label learning model with as few labeled examples as possible. Active learning, which actively selects the most valuable data to query their labels, is the most important approach to reduce labeling cost. In this paper, we propose a novel approach MADM for batch mode multi-label active learning. On one hand, MADM exploits representativeness and diversity in both the feature and label space by matching the distribution between labeled and unlabeled data. On the other hand, it tends to query predicted positive instances, which are expected to be more informative than negative ones. Experiments on benchmark datasets demonstrate that the proposed approach can reduce the labeling cost significantly. [ABSTRACT FROM AUTHOR]

Many brain disorders and diseases exhibit heterogeneous symptoms and imaging characteristics. This heterogeneity is typically not captured by commonly adopted neuroimaging analyses that seek only a main imaging pattern when two groups need to be differentiated (e.g., patients and controls, or clinical progressors and non-progressors). We propose a novel probabilistic clustering approach, CHIMERA, modeling the pathological process by a combination of multiple regularized transformations from normal/control population to the patient population, thereby seeking to identify multiple imaging patterns that relate to disease effects and to better characterize disease heterogeneity. In our framework, normal and patient populations are considered as point distributions that are matched by a variant of the coherent point drift algorithm. We explain how the posterior probabilities produced during the MAP optimization of CHIMERA can be used for clustering the patients into groups and identifying disease subtypes. CHIMERA was first validated on a synthetic dataset and then on a clinical dataset mixing 317 control subjects and patients suffering from Alzheimer's Disease (AD) and Parkison's Disease (PD). CHIMERA produced better clustering results compared to two standard clustering approaches. We further analyzed 390 T1 MRI scans from Alzheimer's patients. We discovered two main and reproducible AD subtypes displaying significant differences in cognitive performance. [ABSTRACT FROM PUBLISHER]

Distribution matching transforms independent and Bernoulli(1/2) distributed input bits into a sequence of output symbols with a desired distribution. Fixed-to-fixed length, invertible, and low complexity encoders and decoders based on constant composition and arithmetic coding are presented. The encoder achieves the maximum rate, namely, the entropy of the desired distribution, asymptotically in the blocklength. Furthermore, the normalized divergence of the encoder output and the desired distribution goes to zero in the blocklength. [ABSTRACT FROM PUBLISHER]

This paper describes a novel two-stage dereverberation feature enhancement method for noise-robust automatic speech recognition. In the first stage, an estimate of the dereverberated speech is generated by matching the distribution of the observed reverberant speech to that of clean speech, in a decorrelated transformation domain that has a long temporal context in order to address the effects of reverberation. The second stage uses this dereverberated signal as an initial estimate within a non-negative matrix factorization framework, which jointly estimates a sparse representation of the clean speech signal and an estimate of the convolutional distortion. The proposed feature enhancement method, when used in conjunction with automatic speech recognizer back-end processing, is shown to improve the recognition performance compared to three other state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

In a standardk-nearest neighbor (kNN) technique, imputations of unit-level values in the variables of interest (Y) are based on thek-nearest neighbors in a set of reference units. Nearest is defined with respect to a distance metric in the space of auxiliary variables (X). This study evaluateskNN imputations ofYwith a selection, by the same distance metric, ofk-nearest locally weighted regression models. Imputations are obtained as predictions using theXvalues of thek-nearest neighbors in the population. In simulated random sampling from three artificial multivariate populations and two actual univariate populations and sampling units composed of a single population element or a cluster of four elements, the newkNN technique: (1) improved the correlation between an imputation and its actual value; (2) lowered the root mean square error (RMSE) of imputations; (3) increased the slope in regressions of actualyvalues regressed against their imputed values; (4) performed relatively best withkvalues of 4 and sample sizes of 200 or greater; (5) compared favorably with a recently proposedkNN calibration procedure; and (6) had a higher (15–28%) RMSE than with a simple local linear regression. Distribution matching had a consistent negative effect (+10%) on RMSE. [ABSTRACT FROM AUTHOR]

This study investigates a convex relaxation approach to figure-ground separation with a global distribution matching prior evaluated by the Bhattacharyya measure. The problem amounts to finding a region that most closely matches a known model distribution. It has been previously addressed by curve evolution, which leads to suboptimal and computationally intensive algorithms, or by graph cuts, which result in metrication errors. Solving a sequence of convex subproblems, the proposed relaxation is based on a novel bound of the Bhattacharyya measure which yields an algorithm robust to initial conditions. Furthermore, we propose a novel flow configuration that accounts for labeling-function variations, unlike existing configurations. This leads to a new max-flow formulation which is dual to the convex relaxed subproblems we obtained. We further prove that such a formulation yields exact and global solutions to the original, nonconvex subproblems. A comprehensive experimental evaluation on the Microsoft GrabCut database demonstrates that our approach yields improvements in optimality and accuracy over related recent methods. [ABSTRACT FROM AUTHOR]

Domain adaptation addresses the prediction problem in which the source and target data are sampled from different but related probability distributions. The key problem here lies in properly matching the distributions and learning general feature representation for training the prediction model. In this article, we introduce a Domain Invariant and Agnostic Adaptation (DIAA) solution, which matches the source and target joint distributions, and simultaneously aligns the feature and domain label joint distribution to its marginal product. In particular, DIAA matches and aligns the distributions via a feature transformation, and compares the two kinds of distribution disparities uniformly under the Kullback–Leibler (KL) divergence. To approximate the two corresponding KL divergences from observed samples, we derive a linear-regression-like technique that fits linear models to different ratio functions under the quadratic loss. With the estimated KL divergences, learning the DIAA feature transformation is formulated as solving a Grassmannian minimization problem. Experiments on text and image classification tasks with varied nature demonstrate the success of our approach. [ABSTRACT FROM AUTHOR]

A novel rotation and scale invariant texture classification methodology is proposed based on distribution matching in higher dimensional space. Feature extraction is performed by using uniform local binary patterns (uLBPs) in which the rotation and scale changes in an image cause shifts in the underlying uLBP histograms. To compensate for these shifts at the classification layer, the distributions of training and testing data using kernel methods are estimated and means of the two distributions in the transformed domain using importance weights are matched. These calculated importance weights are used in the standard support vector machines to compensate for the shift in the distributions. The proposed method is used for classifying the images in the Brodatz texture database demonstrating the effectiveness of the proposed methodology. [ABSTRACT FROM AUTHOR]

In this paper, we provide a systematic comparison of distribution matching (DM) and sphere shaping (SpSh) algorithms for short blocklength probabilistic amplitude shaping. For asymptotically large blocklengths, constant composition distribution matching (CCDM) is known to generate the target capacity-achieving distribution. However, as the blocklength decreases, the resulting rate loss diminishes the efficiency of CCDM. We claim that for such short blocklengths over the additive white Gaussian noise (AWGN) channel, the objective of shaping should be reformulated as obtaining the most energy-efficient signal space for a given rate (rather than matching distributions). In light of this interpretation, multiset-partition DM (MPDM) and SpSh are reviewed as energy-efficient shaping techniques. Numerical results show that both have smaller rate losses than CCDM. SpSh-whose sole objective is to maximize the energy efficiency-is shown to have the minimum rate loss amongst all, which is particularly apparent for ultra short blocklengths. We provide simulation results of the end-to-end decoding performance showing that up to 1 dB improvement in power efficiency over uniform signaling can be obtained with MPDM and SpSh at blocklengths around 200. Finally, we present a discussion on the complexity of these algorithms from the perspectives of latency, storage and computations.