Search

This paper provides a dual domain derivation of the error exponent of maximum mutual information (MMI) decoding with constant composition codes, showing it coincides with that of maximum likelihood decoding for discrete memoryless channels. The analysis is further extended to joint source-channel coding, demonstrating that the generalized MMI decoder achieves the same random coding error exponent as the maximum a posteriori decoder., Comment: This paper has been submitted to ISIT 2025 for review

We show that for a wide range of channels and code ensembles with pairwise-independent codewords, with probability tending to 1 with the code length, expurgating an arbitrarily small fraction of codewords from a randomly selected code results in a code attaining the expurgated exponent.

We find the exact typical error exponent of constant composition generalized random Gilbert-Varshamov (RGV) codes over DMCs channels with generalized likelihood decoding. We show that the typical error exponent of the RGV ensemble is equal to the expurgated error exponent, provided that the RGV codebook parameters are chosen appropriately. We also prove that the random coding exponent converges in probability to the typical error exponent, and the corresponding non-asymptotic concentration rates are derived. Our results show that the decay rate of the lower tail is exponential while that of the upper tail is double exponential above the expurgated error exponent. The explicit dependence of the decay rates on the RGV distance functions is characterized., Comment: 60 pages, 2 figures

We propose a universal classifier for binary Neyman-Pearson classification where null distribution is known while only a training sequence is available for the alternative distribution. The proposed classifier interpolates between Hoeffding's classifier and the likelihood ratio test and attains the same error probability prefactor as the likelihood ratio test, i.e., the same prefactor as if both distributions were known. In addition, like Hoeffding's universal hypothesis test, the proposed classifier is shown to attain the optimal error exponent tradeoff attained by the likelihood ratio test whenever the ratio of training to observation samples exceeds a certain value. We propose a lower bound and an upper bound to the training to observation ratio. In addition, we propose a sequential classifier that attains the optimal error exponent tradeoff.

This paper shows that the probability that the error exponent of a given code randomly generated from a pairwise independent ensemble being smaller than a lower bound on the typical random-coding exponent tends to zero as the codeword length tends to infinity. This lower bound is known to be tight for i.i.d. ensembles over the binary symmetric channel and for constant-composition codes over memoryless channels. Our results recover both as special cases and remain valid for arbitrary alphabets, arbitrary channels -- for example finite-state channels with memory -- and arbitrary pairwise-independent ensembles. We specialize our results to the i.i.d., constant-composition and cost-constrained ensembles over discrete memoryless channels and to ensembles over finite-state channels., Comment: 31 pages, 1 figure

This paper studies the concentration properties of random codes. Specifically, we show that, for discrete memoryless channels, the error exponent of a randomly generated code with pairwise-independent codewords converges in probability to its expectation -- the typical error exponent. For high rates, the result is a consequence of the fact that the random-coding error exponent and the sphere-packing error exponent coincide. For low rates, instead, the convergence is based on the fact that the union bound accurately characterizes the probability of error. The paper also zooms into the behavior at asymptotically low rates and shows that the error exponent converges in distribution to a Gaussian-like distribution. Finally, we present several results on the convergence of the error probability and error exponent for generic ensembles and channels., Comment: 98 pages

We derive a sphere-packing error exponent for coded transmission over discrete memoryless channels with a fixed decoding metric. By studying the error probability of the code over an auxiliary channel, we find a lower bound to the probability of error of mismatched decoding. The bound is shown to decay exponentially for coding rates smaller than a new upper bound to the mismatch capacity which is established in this paper. For rates higher than the new upper bound, the error probability is shown to be bounded away from zero. The new upper bound is shown to improve over previous upper bounds to the mismatch capacity.

We study a variation of Bayesian M-ary hypothesis testing in which the test outputs a list of L candidates out of the M possible upon processing the observation. We study the minimum error probability of list hypothesis testing, where an error is defined as the event where the true hypothesis is not in the list output by the test. We derive two exact expressions of the minimum probability or error. The first is expressed as the error probability of a certain non-Bayesian binary hypothesis test, and is reminiscent of the meta-converse bound. The second, is expressed as the tail probability of the likelihood ratio between the two distributions involved in the aforementioned non-Bayesian binary hypothesis test., Comment: 11 pages, submitted to Information and Inference: a journal of the IMA

We study the problem of mismatched binary hypothesis testing between i.i.d. distributions. We analyze the tradeoff between the pairwise error probability exponents when the actual distributions generating the observation are different from the distributions used in the likelihood ratio test, sequential probability ratio test, and Hoeffding's generalized likelihood ratio test in the composite setting. When the real distributions are within a small divergence ball of the test distributions, we find the deviation of the worst-case error exponent of each test with respect to the matched error exponent. In addition, we consider the case where an adversary tampers with the observation, again within a divergence ball of the observation type. We show that the tests are more sensitive to distribution mismatch than to adversarial observation tampering., Comment: arXiv admin note: text overlap with arXiv:2001.03917

We derive an upper bound on the reliability function of mismatched decoding for zero-rate codes. The bound is based on a result by Koml\'os that shows the existence of a subcode with certain symmetry properties. The bound is shown to coincide with the expurgated exponent at rate zero for a broad family of channel-decoding metric pairs.

We derive a single-letter upper bound to the mismatched-decoding capacity for discrete memoryless channels. The bound is expressed as the mutual information of a transformation of the channel, such that a maximum-likelihood decoding error on the translated channel implies a mismatched-decoding error in the original channel. In particular, a strong converse is shown to hold for this upper-bound: if the rate exceeds the upper-bound, the probability of error tends to 1 exponentially when the block-length tends to infinity. We also show that the underlying optimization problem is a convex-concave problem and that an efficient iterative algorithm converges to the optimal solution. In addition, we show that, unlike achievable rates in the literature, the multiletter version of the bound does not improve. A number of examples are discussed throughout the paper.

Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical considerations such as channel uncertainty and implementation constraints rule out the use of an optimal decoder. The mismatched decoding problem addresses such scenarios by considering the case that the decoder cannot be optimized, but is instead fixed as part of the problem statement. This problem is not only of direct interest in its own right, but also has close connections with other long-standing theoretical problems in information theory. In this monograph, we survey both classical literature and recent developments on the mismatched decoding problem, with an emphasis on achievable random-coding rates for memoryless channels. We present two widely-considered achievable rates known as the generalized mutual information (GMI) and the LM rate, and overview their derivations and properties. In addition, we survey several improved rates via multi-user coding techniques, as well as recent developments and challenges in establishing upper bounds on the mismatch capacity, and an analogous mismatched encoding problem in rate-distortion theory. Throughout the monograph, we highlight a variety of applications and connections with other prominent information theory problems., Comment: Published in Foundations and Trends in Communications and Information Theory (Volume 17, Issue 2-3)

We study the problem of mismatched likelihood ratio test. We analyze the type-\RNum{1} and \RNum{2} error exponents when the actual distributions generating the observation are different from the distributions used in the test. We derive the worst-case error exponents when the actual distributions generating the data are within a relative entropy ball of the test distributions. In addition, we study the sensitivity of the test for small relative entropy balls.

This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which coincide in some cases. In addition, the probability of randomly drawing a codebook whose error exponent is larger than the TRC exponent is shown to be double-exponentially small; upper and lower bounds to the double-exponential exponent are given. The results suggest that codebooks whose error exponent is larger than the error exponent of the TRC are extremely rare. The key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables.

Motivated by the sequence reconstruction problem from traces in DNA-based storage, we consider the problem of designing codes for the deletion channel when multiple observations (or traces) are available to the decoder. We propose simple binary and non-binary codes based on Varshamov-Tenengolts (VT) codes. The proposed codes split the codeword in blocks and employ a VT code in each block. The availability of multiple traces helps the decoder to identify deletion-free copies of a block, and to avoid mis-synchronization while decoding. The encoding complexity of the proposed scheme is linear in the codeword length; the decoding complexity is linear in the codeword length, and quadratic in the number of deletions and the number of traces. The proposed scheme offers an explicit low-complexity technique for correcting deletions using multiple traces., Comment: This paper will be presented at ISIT 2019

This paper studies the random-coding exponent of joint source-channel coding for the multiple-access channel with correlated sources. For each user, by defining a threshold, the messages of each source are partitioned into two classes. The achievable exponent for correlated sources with two message-dependent input distributions for each user is determined and shown to be larger than that achieved using only one input distribution for each user. A system of equations is presented to determine the optimal thresholds maximizing the achievable exponent. The obtained exponent is compared with the one derived for the MAC with independent sources.

In this paper, an achievable error exponent for the multiple-access channel with two independent sources is derived. For each user, the source messages are partitioned into two classes and codebooks are generated by drawing codewords from an input distribution depending on the class index of the source message. The partitioning thresholds that maximize the achievable exponent are given by the solution of a system of equations. We also derive both lower and upper bounds for the achievable exponent in terms of Gallager's source and channel functions. Finally, a numerical example shows that using the proposed ensemble gives a noticeable gain in terms of exponent with respect to independent identically distributed codebooks.

We introduce a definition of perfect and quasi-perfect codes for symmetric channels parametrized by an auxiliary output distribution. This notion generalizes previous definitions of perfect and quasi-perfect codes and encompasses maximum distance separable codes. The error probability of these codes, whenever they exist, is shown to coincide with the estimate provided by the meta-converse lower bound. We illustrate how the proposed definition naturally extends to cover almost-lossless source-channel coding and lossy compression., Comment: Submitted to IEEE Transactions on Information Theory

We introduce a random coding technique for transmission over discrete memoryless channels, reminiscent of the basic construction attaining the Gilbert-Varshamov bound for codes in Hamming spaces. The code construction is based on drawing codewords recursively from a fixed type class, in such a way that a newly generated codeword must be at a certain minimum distance from all previously chosen codewords, according to some generic distance function. We derive an achievable error exponent for this construction, and prove its tightness with respect to the ensemble average. We show that the exponent recovers the Csisz\'{a}r and K{\"o}rner exponent as a special case, which is known to be at least as high as both the random-coding and expurgated exponents, and we establish the optimality of certain choices of the distance function. In addition, for additive distances and decoding metrics, we present an equivalent dual expression, along with a generalization to infinite alphabets via cost-constrained random coding., Comment: Simplified proofs compared to previous version. Final version for IEEE Transactions on Information Theory

Varshamov-Tenengolts (VT) codes are a class of codes which can correct a single deletion or insertion with a linear-time decoder. This paper addresses the problem of efficient encoding of non-binary VT codes, defined over an alphabet of size $q >2$. We propose a simple linear-time encoding method to systematically map binary message sequences onto VT codewords. The method provides a new lower bound on the size of $q$-ary VT codes of length $n$., Comment: This paper will appear in the proceedings of ISIT 2018

Consider two remote nodes (encoder and decoder), each with a binary sequence. The encoder's sequence $X$ differs from the decoder's sequence $Y$ by a small number of edits (deletions and insertions). The goal is to construct a message $M$, to be sent via a one-way error free link, such that the decoder can reconstruct $X$ using $M$ and $Y$. In this paper, we devise a coding scheme for this one-way synchronization model. The scheme is based on multiple layers of Varshamov-Tenengolts (VT) codes combined with off-the-shelf linear error-correcting codes, and uses a list decoder. We bound the expected list size of the decoder under certain assumptions, and validate its performance via numerical simulations. We also consider an alternative decoder that uses only the constraints from the VT codes (i.e., does not require a linear code), and has a smaller redundancy at the expense of a slightly larger average list size., Comment: This paper is accepted for publication in the IEEE Transactions on Information Theory

This paper considers insertion and deletion channels with the additional assumption that the channel input sequence is implicitly divided into segments such that at most one edit can occur within a segment. No segment markers are available in the received sequence. We propose code constructions for the segmented deletion, segmented insertion, and segmented insertion-deletion channels based on subsets of Varshamov-Tenengolts codes chosen with pre-determined prefixes and/or suffixes. The proposed codes, constructed for any finite alphabet, are zero-error and can be decoded segment-by-segment. We also derive an upper bound on the rate of any zero-error code for the segmented edit channel, in terms of the segment length. This upper bound shows that the rate scaling of the proposed codes as the segment length increases is the same as that of the maximal code., Comment: Appeared in IEEE Transactions on Information Theory

This paper studies channel coding for the discrete memoryless multiple-access channel with a given (possibly suboptimal) decoding rule. A multi-letter successive decoding rule depending on an arbitrary non-negative decoding metric is considered, and achievable rate regions and error exponents are derived both for the standard MAC (independent codebooks), and for the cognitive MAC (one user knows both messages) with superposition coding. In the cognitive case, the rate region and error exponent are shown to be tight with respect to the ensemble average. The rate regions are compared with those of the commonly-considered decoder that chooses the message pair maximizing the decoding metric, and numerical examples are given for which successive decoding yields a strictly higher sum rate for a given pair of input distributions., Comment: Accepted to IEEE Transactions on Information Theory

This paper studies the mismatched decoding problem for binary-input discrete memoryless channels. An example is provided for which an achievable rate based on superposition coding exceeds the LM rate (Hui, 1983; Csisz\'ar-K\"orner, 1981), thus providing a counter-example to a previously reported converse result (Balakirsky, 1995). Both numerical evaluations and theoretical results are used in establishing this claim., Comment: Extended version of paper accepted to IEEE Transactions on Information Theory; rate derivation and numerical algorithms included in appendices

Two alternative exact characterizations of the minimum error probability of Bayesian M-ary hypothesis testing are derived. The first expression corresponds to the error probability of an induced binary hypothesis test and implies the tightness of the meta-converse bound by Polyanskiy, Poor and Verd\'u; the second expression is function of an information-spectrum measure and implies the tightness of a generalized Verd\'u-Han lower bound. The formulas characterize the minimum error probability of several problems in information theory and help to identify the steps where existing converse bounds are loose., Comment: Accepted for publication in the IEEE Transactions on Information Theory

This paper studies an almost-lossless source-channel coding scheme in which source messages are assigned to different classes and encoded with a channel code that depends on the class index. The code performance is analyzed by means of random-coding error exponents and validated by simulation of a low-complexity implementation using existing source and channel codes. While each class code can be seen as a concatenation of a source code and a channel code, the overall performance improves on that of separate source-channel coding and approaches that of joint source-channel coding when the number of classes increases., Comment: Published in the IEEE Transactions on Information Theory

We consider a multi-input multi-output (MIMO) block-fading channel with a general model for channel state information at the transmitter (CSIT). The model covers systems with causal CSIT, where only CSIT of past fading blocks is available, and predictive CSIT, where CSIT of some future fading blocks is available. The optimal diversity-multiplexing tradeoff (DMT) and rate-diversity tradeoff (RDT) of the channel are studied under long-term power constraints. The impact of imperfect (mismatched) CSIT on the optimal DMT and RDT is also investigated. Our results show the outage diversity gain obtained by providing imperfect causal/predictive CSIT, leading to new insights into system design and analysis., Comment: Extended version of ISIT 2010 paper. 23 pages, 8 figures

This paper presents a saddlepoint approximation of the random-coding union bound of Polyanskiy et al. for i.i.d. random coding over discrete memoryless channels. The approximation is single-letter, and can thus be computed efficiently. Moreover, it is shown to be asymptotically tight for both fixed and varying rates, unifying existing achievability results in the regimes of error exponents, second-order coding rates, and moderate deviations. For fixed rates, novel exact-asymptotics expressions are specified to within a multiplicative 1+o(1) term. A numerical example is provided for which the approximation is remarkably accurate even at short block lengths., Comment: Accepted to ISIT 2014, presented without publication at ITA 2014

This paper studies multiuser random coding techniques for channel coding with a given (possibly suboptimal) decoding rule. For the mismatched discrete memoryless multiple-access channel, an error exponent is obtained that is tight with respect to the ensemble average, and positive within the interior of Lapidoth's achievable rate region. This exponent proves the ensemble tightness of the exponent of Liu and Hughes in the case of maximum-likelihood decoding. An equivalent dual form of Lapidoth's achievable rate region is given, and the latter is shown to extend immediately to channels with infinite and continuous alphabets. In the setting of single-user mismatched decoding, similar analysis techniques are applied to a refined version of superposition coding, which is shown to achieve rates at least as high as standard superposition coding for any set of random-coding parameters., Comment: Accepted to IEEE Transactions on Information Theory

This paper studies expurgated random-coding bounds and exponents for channel coding with a given (possibly suboptimal) decoding rule. Variations of Gallager's analysis are presented, yielding several asymptotic and non-asymptotic bounds on the error probability for an arbitrary codeword distribution. A simple non-asymptotic bound is shown to attain an exponent of Csisz\'ar and K\"orner under constant-composition coding. Using Lagrange duality, this exponent is expressed in several forms, one of which is shown to permit a direct derivation via cost-constrained coding which extends to infinite and continuous alphabets. The method of type class enumeration is studied, and it is shown that this approach can yield improved exponents and better tightness guarantees for some codeword distributions. A generalization of this approach is shown to provide a multi-letter exponent which extends immediately to channels with memory. Finally, a refined analysis expurgated i.i.d. random coding is shown to yield a O\big(\frac{1}{\sqrt{n}}\big) prefactor, thus improving on the standard O(1) prefactor. Moreover, the implied constant is explicitly characterized., Comment: (v1) Submitted to IEEE Transactions on Information Theory (v2) Extended version of the revision submitted to IEEE Transactions on Information Theory (v3) Extended version of final paper to appear in IEEE Transactions on Information Theory

This paper studies the subexponential prefactor to the random-coding bound for a given rate. Using a refinement of Gallager's bounding techniques, an alternative proof of a recent result by Altu\u{g} and Wagner is given, and the result is extended to the setting of mismatched decoding., Comment: Published at Allerton Conference 2013. (v3) Final version uploaded

This paper studies the random-coding exponent of joint source-channel coding for a scheme where source messages are assigned to disjoint subsets (referred to as classes), and codewords are independently generated according to a distribution that depends on the class index of the source message. For discrete memoryless systems, two optimally chosen classes and product distributions are found to be sufficient to attain the sphere-packing exponent in those cases where it is tight., Comment: Submitted to IEEE Transactions on Information Theory

This paper considers the problem of channel coding with a given (possibly suboptimal) maximum-metric decoding rule. A cost-constrained random-coding ensemble with multiple auxiliary costs is introduced, and is shown to achieve error exponents and second-order coding rates matching those of constant-composition random coding, while being directly applicable to channels with infinite or continuous alphabets. The number of auxiliary costs required to match the error exponents and second-order rates of constant-composition coding is studied, and is shown to be at most two. For i.i.d. random coding, asymptotic estimates of two well-known non-asymptotic bounds are given using saddlepoint approximations. Each expression is shown to characterize the asymptotic behavior of the corresponding random-coding bound at both fixed and varying rates, thus unifying the regimes characterized by error exponents, second-order rates and moderate deviations. For fixed rates, novel exact asymptotics expressions are obtained to within a multiplicative 1+o(1) term. Using numerical examples, it is shown that the saddlepoint approximations are highly accurate even at short block lengths., Comment: Accepted to IEEE Transactions on Information Theory. (v2) Major revisions made, Saddlepoint Approximation section extended significantly, title changed (v3) Final version uploaded

This paper studies the second-order asymptotics of coding rates for the discrete memoryless multiple-access channel with a fixed target error probability. Using constant-composition random coding, coded time-sharing, and a variant of Hoeffding's combinatorial central limit theorem, an inner bound on the set of locally achievable second-order coding rates is given for each point on the boundary of the capacity region. It is shown that the inner bound for constant-composition random coding includes that recovered by i.i.d. random coding, and that the inclusion may be strict. The inner bound is extended to the Gaussian multiple-access channel via an increasingly fine quantization of the inputs., Comment: (v2) Results/proofs given in matrix notation, det(V)=0 handled more rigorously, Berry-Esseen derivation given. (v3) Gaussian case added (v4) Significant change of presentation; added local dispersion results; added new method to obtain non-standard tangent vector terms using coded time-sharing; (v5) Final version (IEEE Transactions on Information Theory)

We study the information rates of non-coherent, stationary, Gaussian, multiple-input multiple-output (MIMO) flat-fading channels that are achievable with nearest neighbor decoding and pilot-aided channel estimation. In particular, we investigate the behavior of these achievable rates in the limit as the signal- to-noise ratio (SNR) tends to infinity by analyzing the capacity pre-log, which is defined as the limiting ratio of the capacity to the logarithm of the SNR as the SNR tends to infinity. We demonstrate that a scheme estimating the channel using pilot symbols and detecting the message using nearest neighbor decoding (while assuming that the channel estimation is perfect) essentially achieves the capacity pre-log of non-coherent multiple-input single-output flat-fading channels, and it essentially achieves the best so far known lower bound on the capacity pre-log of non-coherent MIMO flat-fading channels. We then extend our analysis to the multiple-access channel., Comment: 47 pages, 5 figures. arXiv admin note: text overlap with arXiv:1107.1640, arXiv:1103.0205

This paper determines the range of feasible values of standard error exponents for binary-input memoryless symmetric channels of fixed capacity $C$ and shows that extremes are attained by the binary symmetric and the binary erasure channel. The proof technique also provides analogous extremes for other quantities related to Gallager's $E_0$ function, such as the cutoff rate, the Bhattacharyya parameter, and the channel dispersion., Comment: 6 pages, 4 figures, accepted IEEE Transactions on Information Theory

We determine the loss in capacity incurred by using signal constellations with a bounded support over general complex-valued additive-noise channels for suitably high signal-to-noise ratio. Our expression for the capacity loss recovers the power loss of 1.53dB for square signal constellations., Comment: To appear in the Proceedings of the 2012 IEEE International Symposium on Information Theory (ISIT). Corrected a minor error in Figure 1

We study a noncoherent multiple-input multiple-output (MIMO) fading multiple-access channel (MAC), where the transmitters and the receiver are aware of the statistics of the fading, but not of its realisation. We analyse the rate region that is achievable with nearest neighbour decoding and pilot-assisted channel estimation and determine the corresponding pre-log region, which is defined as the limiting ratio of the rate region to the logarithm of the SNR as the SNR tends to infinity., Comment: 8 pages. Presented at the Forty-Ninth Annual Allerton Conference on Communication, Control and Computing, Allerton Retreat Center, Monticello, IL, September 28-30, 2011. Corrected some minor typos

We study the information rates of non-coherent, stationary, Gaussian, multiple-input multiple-output (MIMO) flat-fading channels that are achievable with nearest neighbour decoding and pilot-aided channel estimation. In particular, we analyse the behaviour of these achievable rates in the limit as the signal-to-noise ratio (SNR) tends to infinity. We demonstrate that nearest neighbour decoding and pilot-aided channel estimation achieves the capacity pre-log - which is defined as the limiting ratio of the capacity to the logarithm of SNR as the SNR tends to infinity - of non-coherent multiple-input single-output (MISO) flat-fading channels, and it achieves the best so far known lower bound on the capacity pre-log of non-coherent MIMO flat-fading channels., Comment: 5 pages, 1 figure. To be presented at the IEEE International Symposium on Information Theory (ISIT), St. Petersburg, Russia, 2011. Replaced with version that will appear in the proceedings

We analyze multiuser detection under the assumption that the number of users accessing the channel is unknown by the receiver. In this environment, users' activity must be estimated along with any other parameters such as data, power, and location. Our main goal is to determine the performance loss caused by the need for estimating the identities of active users, which are not known a priori. To prevent a loss of optimality, we assume that identities and data are estimated jointly, rather than in two separate steps. We examine the performance of multiuser detectors when the number of potential users is large. Statistical-physics methodologies are used to determine the macroscopic performance of the detector in terms of its multiuser efficiency. Special attention is paid to the fixed-point equation whose solution yields the multiuser efficiency of the optimal (maximum a posteriori) detector in the large signal-to-noise ratio regime. Our analysis yields closed-form approximate bounds to the minimum mean-squared error in this regime. These illustrate the set of solutions of the fixed-point equation, and their relationship with the maximum system load. Next, we study the maximum load that the detector can support for a given quality of service (specified by error probability)., Comment: to appear in IEEE Transactions on Information Theory

We study analogue source transmission over MIMO block-fading channels with receiver-only channel state information. Unlike previous work which considers the end-to-end expected distortion as a figure of merit, we study the distortion outage probability. We first consider the well known transmitter informed bound, which yields a benchmark lower bound to the distortion outage probability of any coding scheme. We next compare the results with source-channel separation. The key difference from the expected distortion approach is that if the channel code rate is chosen appropriately, source-channel separation can not only achieve the same diversity exponent, but also the same distortion outage probability as the transmitter informed lower bound., Comment: 5 pages, 4 figures, Accepted, 2010 IEEE International Symposium on Information Theory (ISIT 2010)

We study the asymptotic outage performance of incremental redundancy automatic repeat request (INR-ARQ) transmission over the multiple-input multiple-output (MIMO) block-fading channels with discrete input constellations. We first show that transmission with random codes using a discrete signal constellation across all transmit antennas achieves the optimal outage diversity given by the Singleton bound. We then analyze the optimal SNR-exponent and outage diversity of INR-ARQ transmission over the MIMO block-fading channel. We show that a significant gain in outage diversity is obtained by providing more than one bit feedback at each ARQ round. Thus, the outage performance of INR-ARQ transmission can be remarkably improved with minimal additional overhead. A suboptimal feedback and power adaptation rule, which achieves the optimal outage diversity, is proposed for MIMO INR-ARQ, demonstrating the benefits provided by multi-bit feedback., Comment: 28 pages, 7 figures; submitted to IEEE Transactions on Information Theory

We study irregular binary turbo codes over non-ergodic block-fading channels. We first propose an extension of channel multiplexers initially designed for regular turbo codes. We then show that, using these multiplexers, irregular turbo codes that exhibit a small decoding threshold over the ergodic Gaussian-noise channel perform very close to the outage probability on block-fading channels, from both density evolution and finite-length perspectives., Comment: to be presented at the IEEE International Symposium on Information Theory, 2010

The performance of bit-interleaved coded modulation (BICM) with bit shaping (i.e., non-equiprobable bit probabilities in the underlying binary code) is studied. For the Gaussian channel, the rates achievable with BICM and bit shaping are practically identical to those of coded modulation or multilevel coding. This identity holds for the whole range of values of signal-to-noise ratio. Moreover, the random coding error exponent of BICM significantly exceeds that of multilevel coding and is very close to that of coded modulation., Comment: 5 pages, submitted to the 2010 IEEE International Symposium on Information Theory

Reliable communication over delay-constrained block-fading channels with discrete inputs and mismatched (imperfect) channel state information at the transmitter (CSIT) is studied. The CSIT mismatch is modeled as Gaussian random variables, whose variances decay as a power of the signal-to-noise ratio (SNR). A special focus is placed on the large-SNR decay of the outage probability when power control with long-term power constraints is used. Without explicitly characterizing the corresponding power allocation algorithms, we derive the outage exponent as a function of the system parameters, including the CSIT noise variance exponent and the exponent of the peak power constraint. It is shown that CSIT, even if noisy, is always beneficial and leads to important gains in terms of exponents. It is also shown that when multidimensional rotations or precoders are used at the transmitter, further exponent gains can be attained, but at the expense of larger decoding complexity., Comment: submitted to the IEEE Transactions on Information Theory. To be presented (in part) at the 2009 IEEE International Symposium on Information Theory, Seoul, South Korea, June-July 2009

We revisit the information-theoretic analysis of bit-interleaved coded modulation (BICM) by modeling the BICM decoder as a mismatched decoder. The mismatched decoding model is well-defined for finite, yet arbitrary, block lengths, and naturally captures the channel memory among the bits belonging to the same symbol. We give two independent proofs of the achievability of the BICM capacity calculated by Caire et al. where BICM was modeled as a set of independent parallel binary-input channels whose output is the bitwise log-likelihood ratio. Our first achievability proof uses typical sequences, and shows that due to the random coding construction, the interleaver is not required. The second proof is based on the random coding error exponents with mismatched decoding, where the largest achievable rate is the generalized mutual information. We show that the generalized mutual information of the mismatched decoder coincides with the infinite-interleaver BICM capacity. We also show that the error exponent -and hence the cutoff rate- of the BICM mismatched decoder is upper bounded by that of coded modulation and may thus be lower than in the infinite-interleaved model. We also consider the mutual information appearing in the analysis of iterative decoding of BICM with EXIT charts. We show that the corresponding symbol metric has knowledge of the transmitted symbol and the EXIT mutual information admits a representation as a pseudo-generalized mutual information, which is in general not achievable. A different symbol decoding metric, for which the extrinsic side information refers to the hypothesized symbol, induces a generalized mutual information lower than the coded modulation capacity., Comment: submitted to the IEEE Transactions on Information Theory. Conference version in 2008 IEEE International Symposium on Information Theory, Toronto, Canada, July 2008

The free-space optical channel has the potential to facilitate inexpensive, wireless communication with fiber-like bandwidth under short deployment timelines. However, atmospheric effects can significantly degrade the reliability of a free-space optical link. In particular, atmospheric turbulence causes random fluctuations in the irradiance of the received laser beam, commonly referred to as scintillation. The scintillation process is slow compared to the large data rates typical of optical transmission. As such, we adopt a quasi-static block fading model and study the outage probability of the channel under the assumption of orthogonal pulse-position modulation. We investigate the mitigation of scintillation through the use of multiple lasers and multiple apertures, thereby creating a multiple-input multiple output (MIMO) channel. Non-ideal photodetection is also assumed such that the combined shot noise and thermal noise are considered as signal-independent Additive Gaussian white noise. Assuming perfect receiver channel state information (CSI), we compute the signal-to-noise ratio exponents for the cases when the scintillation is lognormal, exponential and gamma-gamma distributed, which cover a wide range of atmospheric turbulence conditions. Furthermore, we illustrate very large gains, in some cases larger than 15 dB, when transmitter CSI is also available by adapting the transmitted electrical power., Comment: 24 pages, 3 figures, 2 tables. Submitted to the IEEE Journal of Lightwave Technology

Power allocation with peak-to-average power ratio constraints is investigated for transmission over Nakagami-m fading channels with arbitrary input distributions. In the case of delay-limited block-fading channels, we find the solution to the minimum outage power allocation scheme with peak-to-average power constraints and arbitrary input distributions, and show that the signal-to-noise ratio exponent for any finite peak-to-average power ratio is the same as that of the peak-power limited problem, resulting in an error floor. In the case of the ergodic fully-interleaved channel, we find the power allocation rule that yields the maximal information rate for an arbitrary input distribution and show that capacities with peak-to-average power ratio constraints, even for small ratios, are very close to capacities without peak-power restrictions., Comment: 26 pages, 6 figures, submitted to IEEE Transaction on Wireless Communication