Search

In this manuscript we define the notion of "$\delta$-typicality" for both entropy and relative entropy, as well as a notion of $\epsilon$-goodness and provide an extension to Stein's lemma for continuous quantities as well as correlated setups. We apply the derived results on the Gaussian hypothesis testing problem where the observations are possibly correlated., Comment: 11 pages

We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed $\alpha$-stable sources ($0 < \alpha < 2$). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function for $\alpha$-stable sources subject to a constraint on the strength of the error, and show it to be logarithmic in the strength-to-distortion ratio. We showcase how our framework paves the way to finding optimal quantizers for $\alpha$-stable sources and more generally to heavy-tailed ones. In addition, we study high-rate scalar quantizers and show that uniform ones are asymptotically optimal under the strength measure. We compare uniform Gaussian and Cauchy quantizers and show that more representation points for the Cauchy source are required to guarantee the same quantization quality. Our findings generalize the well-known rate-distortion and quantization results of Gaussian sources ($\alpha = 2$) under a quadratic distortion measure., Comment: 28 pages, 6 figures

Isotropic $\alpha$-stable distributions are central in the theory of heavy-tailed distributions and play a role similar to that of the Gaussian density among finite second-moment laws. Given a sequence of $n$ observations, we are interested in characterizing the performance of Likelihood Ratio Tests where two hypotheses are plausible for the observed quantities: either isotropic Cauchy or isotropic Gaussian. Under various setups, we show that the probability of error of such detectors is not always exponentially decaying with $n$ with the leading term in the exponent shown to be logarithmic instead and we determine the constants in that leading term. Perhaps surprisingly, the optimal Bayesian probabilities of error are found to exhibit different asymptotic behaviors.

The per-sample zero-dispersion channel model of the optical fiber is considered. It is shown that capacity is uniquely achieved by an input probability distribution that has continuous uniform phase and discrete amplitude that takes on finitely many values. This result holds when the channel is subject to general input cost constraints, that include a peak amplitude constraint and a joint average and peak amplitude constraint., Comment: 18 pages; Submitted for review to the IEEE Transactions on Information Theory

Recent studies found that many channels are affected by additive noise that is impulsive in nature and is best explained by heavy-tailed symmetric alpha-stable distributions. Dealing with impulsive noise environments comes with an added complexity with respect to the standard Gaussian environment: the alpha-stable probability density functions have an infinite second moment and the "nice" Hilbert space structure of the space of random variables having a finite second moment is lost along with its tools and methodologies. This is indeed the case in estimation theory where classical tools to quantify performance of an estimator are tightly related to the assumption of finite variance variables. In alpha-stable environments, expressions such as the mean square error and the Cramer-Rao bound are hence problematic. In this work, we tackle the parameter estimation problem in impulsive noise environments and develop novel tools that are tailored to the alpha-stable and heavy-tailed noise environments, tools that coincide with the standard ones adopted in the Gaussian setup, namely a generalized "power" measure and a generalized Fisher information. We generalize known information inequalities commonly used in the Gaussian context: the de Bruijn's identity, the data processing inequality, the Fisher information inequality, the isoperimetric inequality for entropies and the Cramer-Rao bound. Additionally, we derive upper bounds on the differential entropy of independent sums having a stable component. Finally, the new "power" measure is used to shed some light on the additive alpha-stable noise channel capacity in a setup that generalizes the linear average power constrained AWGN channel. Our theoretical findings are paralleled with numerical evaluations of various quantities and bounds using developed {\em Matlab} packages., Comment: 42 pages, 5 figures, submitted to the IEEE transactions on information theory for peer review

In his technical report~\cite[sec. 6]{barrontech}, Barron states that the de Bruijn's identity for Gaussian perturbations holds for any RV having a finite variance. In this report, we follow Barron's steps as we prove the existence of $J_{\alpha}\left(X + \sqrt[\alpha]{\eta}N\right)$, $\eta > 0$ for any Radom Variable (RV) $X \in \mathcal{L}$ where \begin{equation*} \mathcal{L} = \left\{ \text{RVs} \,\,U: \int \ln\left(1 + |U|\right)\,dF_{U}(u) \text{ is finite } \right\}, \end{equation*} and where $N \sim \mathcal{S}(\alpha;1)$ is independent of $X$, $0< \alpha <2$.

We study the classical problem of characterizing the channel capacity and its achieving distribution in a generic fashion. We derive a simple relation between three parameters: the input-output function, the input cost function and the noise probability density function, one which dictates the type of the optimal input. In Layman terms we prove that the support of the optimal input is bounded whenever the cost grows faster than a cut-off rate equal to the logarithm of the noise PDF evaluated at the input-output function. Furthermore, we prove a converse statement that says whenever the cost grows slower than the cut-off rate, the optimal input has necessarily an unbounded support. In addition, we show how the discreteness of the optimal input is guaranteed whenever the triplet satisfy some analyticity properties. We argue that a suitable cost function to be imposed on the channel input is one that grows similarly to the cut-off rate. Our results are valid for any cost function that is super-logarithmic. They summarize a large number of previous channel capacity results and give new ones for a wide range of communication channel models, such as Gaussian mixtures, generalized-Gaussians and heavy-tailed noise models, that we state along with numerical computations., Comment: Accepted for publication in the IEEE Transactions on Information Theory with minor modifications on the current version

Evaluating the channel capacity is one of many key problems in information theory. In this work we derive rather-mild sufficient conditions under which the capacity is finite and achievable. These conditions are derived for generic, memoryless and possibly non-linear additive noise channels. The results are based on a novel sufficient condition that guarantees the convergence of differential entropies under point-wise convergence of Probability Density Functions. Perhaps surprisingly, the finiteness of channel capacity holds for the majority of setups, including those where inputs and outputs have possibly infinite second-moments., Comment: Submitted to the IEEE Transactions on Communications

Interference shapes the interplay between capacity and coverage in cellular networks. However, interference is non-deterministic and depends on various system and channel parameters including user scheduling, frequency reuse, and fading variations. We present an analytical approach for modeling the distribution of intercell interference in the downlink of cellular networks as a function of generic fading channel models and various scheduling schemes. We demonstrate the usefulness of the derived expressions in calculating location-based and average-based data rates in addition to capturing practical tradeoffs between cell capacity and coverage in downlink cellular networks., Comment: 5 pages, 7 figures, conference