Search

We study the list decodability of different ensembles of codes over the real alphabet under the assumption of an omniscient adversary. It is a well-known result that when the source and the adversary have power constraints $P $ and $N $ respectively, the list decoding capacity is equal to $\frac {1}{2}\log \frac {P}{N}$. Random spherical codes achieve constant list sizes, and the goal of the present paper is to obtain a better understanding of the smallest achievable list size as a function of the gap to capacity. We show a reduction from arbitrary codes to spherical codes, and derive a lower bound on the list size of typical random spherical codes. We also give an upper bound on the list size achievable using nested Construction-A lattices and infinite Construction-A lattices. We then define and study a class of infinite constellations that generalize Construction-A lattices and prove upper and lower bounds for the same. Other goodness properties such as packing goodness and AWGN goodness of infinite constellations are proved along the way. Finally, we consider random lattices sampled from the Haar distribution and show that if a certain conjecture that originates in analytic number theory is true, then the list size grows as a polynomial function of the gap-to-capacity. [ABSTRACT FROM AUTHOR]

We consider the problem of estimating the mean of a symmetric log-concave distribution under the constraint that only a single bit per sample from this distribution is available to the estimator. We study the mean squared error as a function of the sample size (and hence the number of bits). We consider three settings: first, a centralized setting, where an encoder may release $n$ bits given a sample of size $n$ , and for which there is no asymptotic penalty for quantization; second, an adaptive setting in which each bit is a function of the current observation and previously recorded bits, where we show that the optimal relative efficiency compared to the sample mean is precisely the efficiency of the median; lastly, we show that in a distributed setting where each bit is only a function of a local sample, no estimator can achieve optimal efficiency uniformly over the parameter space. We additionally complement our results in the adaptive setting by showing that one round of adaptivity is sufficient to achieve optimal mean-square error. [ABSTRACT FROM AUTHOR]

In this paper, we consider a problem of sampling a Wiener process, with samples forwarded to a remote estimator over a channel that is modeled as a queue. The estimator reconstructs an estimate of the real-time signal value from causally received samples. We study the optimal online sampling strategy that minimizes the mean square estimation error subject to a sampling rate constraint. We prove that the optimal sampling strategy is a threshold policy, and find the optimal threshold. This threshold is determined by how much the Wiener process varies during the random service time and the maximum allowed sampling rate. Further, if the sampling times are independent of the observed Wiener process, the above sampling problem for minimizing the estimation error is equivalent to a sampling problem for minimizing the age of information. This reveals an interesting connection between the age of information and remote estimation error. Our comparisons show that the estimation error achieved by the optimal sampling policy can be much smaller than those of age-optimal sampling, zero-wait sampling, and periodic sampling. [ABSTRACT FROM AUTHOR]

We show through case studies that it is easier to estimate the fundamental limits of data processing than to construct the explicit algorithms to achieve those limits. Focusing on binary classification, data compression, and prediction under logarithmic loss, we show that in the finite space setting, when it is possible to construct an estimator of the limits with vanishing error with $n$ samples, it may require at least $n\ln n$ samples to construct an explicit algorithm to achieve the limits. [ABSTRACT FROM AUTHOR]

Consider a Gaussian memoryless multiple source (GMMS) with $m$ components with joint probability distribution known only to lie in a given class of distributions. A subset of $k \leq m$ components is sampled and compressed with the objective of reconstructing all the $m$ components within a specified level of distortion under a mean-squared error criterion. In Bayesian and nonBayesian settings, the notion of universal sampling rate-distortion function for Gaussian sources is introduced to capture the optimal tradeoffs among sampling, compression rate, and distortion level. Single-letter characterizations are provided for the universal sampling rate-distortion function. Our achievability proofs highlight the following structural property: it is optimal to compress and reconstruct first the sampled components of the GMMS alone, and then form estimates for the unsampled components based on the former. [ABSTRACT FROM AUTHOR]

Fundamental relations between information and estimation have been established in the literature for the continuous time Gaussian and Poisson channels. In this paper, we demonstrate that such relations hold for a much larger family of continuous-time channels. We introduce the family of semi-martingale channels where the channel output is a semi-martingale stochastic process, and the channel input modulates the characteristics of the semi-martingale. For these channels, which includes as a special case the continuous time Gaussian and Poisson models, we establish new representations relating the mutual information between the channel input and output to an optimal causal filtering loss, thereby unifying and considerably extending results from the Gaussian and Poisson settings. Extensions to the setting of mismatched estimation are also presented where the relative entropy between the laws governing the output of the channel under two different input distributions is equal to the cumulative difference between the estimation loss incurred by using the mismatched and optimal causal filters, respectively. The main tool underlying these results is the Doob–Meyer decomposition of a class of sub-martingales. The results in this paper can be viewed as the continuous-time analogues of recent generalizations for relations between information and estimation for discrete-time Lévy channels. [ABSTRACT FROM AUTHOR]

In this paper, we propose an open-loop unequal-error-protection querying policy based on superposition coding for the noisy 20 questions problem. In this problem, a player wishes to successively refine an estimate of the value of a continuous random variable by posing binary queries and receiving noisy responses. When the queries are designed non-adaptively as a single block and the noisy responses are modeled as the output of a binary symmetric channel, the 20 questions problem can be mapped to an equivalent problem of channel coding with unequal error protection (UEP). A new non-adaptive querying strategy based on UEP superposition coding is introduced, whose estimation error decreases with an exponential rate of convergence that is significantly better than that of the UEP repetition coding introduced by Variani et al. (2015). With the proposed querying strategy, the rate of exponential decrease in the number of queries matches the rate of a closed-loop adaptive scheme, where queries are sequentially designed with the benefit of feedback. Furthermore, the achievable error exponent is significantly better than that of random block codes employing equal error protection. [ABSTRACT FROM AUTHOR]

The problem of modulating the value of a parameter onto a band-limited signal to be transmitted over a continuous-time, additive white Gaussian noise (AWGN) channel, and then estimating this parameter at the receiver, is considered. The performance is measured by the mean power- \alpha error (MP \alpha \text{E} ), which is defined as the worst case \alpha th-order moment of the absolute estimation error. The optimal exponential decay rate of the MP \alpha \textE as a function of the transmission time is investigated. Two upper (converse) bounds on the MP \alpha \textE exponent are derived, on the basis of known bounds for the AWGN channel of inputs with unlimited bandwidth. The bounds are computed for typical values of the error moment and the signal-to-noise ratio (SNR), and the SNR asymptotics of the different bounds are analyzed. The new bounds are compared with known converse and achievability bounds, which were derived from channel coding considerations. [ABSTRACT FROM AUTHOR]

This paper provides new error bounds on consistent reconstruction methods for signals observed from quantized random projections. Those signal estimation techniques guarantee a perfect matching between the available quantized data and a new observation of the estimated signal under the same sensing model. Focusing on dithered uniform scalar quantization of resolution \delta >0 , we prove first that, given a Gaussian random frame of \mathbb R^{N} with M vectors, the worst-case \ell 2 -error of consistent signal reconstruction decays with high probability as O(({N}/{M})\log ({M}/{\sqrt {N}})) uniformly for all signals of the unit ball \mathbb B^N \subset \mathbb R^N . Up to a log factor, this matches a known lower bound in \Omega (N/M) and former empirical validations in O(N/M) . Equivalently, if M exceeds a minimal number of frame coefficients growing like , any vectors in \mathbb B^{N} with M identical quantized projections are at most \epsilon 0 apart with high probability. Second, in the context of quantized compressed sensing with M Gaussian random measurements and under the same scalar quantization scheme, consistent reconstructions of K -sparse signals of \mathbb R^{N} have a worst case error that decreases with high probability as O(({K})/({M})\log ({MN})/({\sqrt {K}^{3}}))$ uniformly for all such signals. Finally, we show that the proximity of vectors whose quantized random projections are only approximately consistent can still be bounded with high probability. A certain level of corruption is thus allowed in the quantization process, up to the appearance of a systematic bias in the reconstruction error of (almost) consistent signal estimates. [ABSTRACT FROM PUBLISHER]

Principal component analysis (PCA) aims at estimating the direction of maximal variability of a high-dimensional data set. A natural question is: does this task become easier, and estimation more accurate, when we exploit additional knowledge on the principal vector? We study the case in which the principal vector is known to lie in the positive orthant. Similar constraints arise in a number of applications, ranging from the analysis of gene expression data to spike sorting in neural signal processing. In the unconstrained case, the estimation performances of PCA have been precisely characterized using the random matrix theory, under a statistical model known as the spiked model. It is known that the estimation error undergoes a phase transition as the signal-to-noise ratio crosses a certain threshold. Unfortunately, tools from the random matrix theory have no bearing on the constrained problem. Despite this challenge, we develop an analogous characterization in the constrained case, within a one-spike model. In particular: 1) we prove that the estimation error undergoes a similar phase transition, albeit at a different thresholds in signal-to-noise ratio that we determine exactly; 2) we prove that—unlike in the unconstrained case—the estimation error depends on the spike vector, and characterize the least favorable vectors; and 3) we show that a non-negative principal component can be approximately computed—under the spiked model—in nearly linear time. This despite the fact that the problem is non-convex and, in general, NP-hard to solve exactly. [ABSTRACT FROM AUTHOR]

We show that the rate regions achieved by linear-feedback coding schemes over dual multi-antenna Gaussian multi-access channels (MACs) and broadcast channels (BCs) with independent noises coincide. By dual here we mean: •

the channel matrices of the MAC and the BC are transposes of each other and [ABSTRACT FROM PUBLISHER]

This paper studies new bounds and code constructions that are applicable to the combinatorial granular channel model previously introduced by Sharov and Roth. We derive new bounds on the maximum cardinality of a grain-error-correcting code and propose constructions of codes that correct grain-errors. We demonstrate that a permutation of the classical group codes (e.g., Constantin–Rao codes) can correct a single grain-error. In many cases of interest, our results improve upon the currently best known bounds and constructions. Some of the approaches adopted in the context of grain-errors may have application to related channel models. [ABSTRACT FROM AUTHOR]

Traditionally, scheduling policies have been optimized to perform well on metrics, such as throughput, delay, and fairness. In the context of shared event schedulers, where a common processor is shared among multiple users, one also has to consider the privacy offered by the scheduling policy. The privacy offered by a scheduling policy measures how much information about the usage pattern of one user of the system can be learned by another as a consequence of sharing the scheduler. We introduced an estimation error-based metric to quantify this privacy. We showed that the most commonly deployed scheduling policy, the first-come-first-served offers very little privacy to its users. We also proposed a parametric nonwork conserving policy, which traded off delay for improved privacy. In this paper, we ask the question, is a tradeoff between delay and privacy fundamental to the design to scheduling policies? In particular, is there a work conserving, possibly randomized, and scheduling policy that scores high on the privacy metric? Answering the first question, we show that there does exist a fundamental limit on the privacy performance of a work-conserving scheduling policy. We quantify this limit. Furthermore, answering the second question, we demonstrate that the round-robin scheduling policy (deterministic policy) is privacy optimal within the class of work-conserving policies. [ABSTRACT FROM AUTHOR]

This paper studies the convergence of the estimation error process and the characterization of the corresponding invariant measure in distributed Kalman filtering for potentially unstable and large linear dynamic systems. A gossip network protocol termed modified gossip interactive Kalman filtering (M-GIKF) is proposed, where sensors exchange their filtered states (estimates and error covariances) and propagate their observations via intersensor communications of rate \bar \gamma ; \bar \gamma is defined as the averaged number of intersensor message passages per signal evolution epoch. The filtered states are interpreted as stochastic particles swapped through local interaction. This paper shows that the conditional estimation error covariance sequence at each sensor under M-GIKF evolves as a random Riccati equation (RRE) with Markov modulated switching. By formulating the RRE as a random dynamical system, it is shown that the network achieves weak consensus, i.e., the conditional estimation error covariance at a randomly selected sensor converges weakly (in distribution) to a unique invariant measure. Further, it is proved that as \bar \gamma \rightarrow \infty this invariant measure satisfies the large deviation (LD) upper and lower bounds, implying that this measure converges exponentially fast (in probability) to the Dirac measure \delta P^{*} , where P^* is the stable error covariance of the centralized (Kalman) filtering setup. The LD results answer a fundamental question on how to quantify the rate at which the distributed scheme approaches the centralized performance as the intersensor communication rate increases. [ABSTRACT FROM PUBLISHER]

This paper establishes a nearly optimal algorithm for denoising a mixture of sinusoids from noisy equispaced samples. We derive our algorithm by viewing line spectral estimation as a sparse recovery problem with a continuous, infinite dictionary. We show how to compute the estimator via semidefinite programming and provide guarantees on its mean-squared error rate. We derive a complementary minimax lower bound on this estimation rate, demonstrating that our approach nearly achieves the best possible estimation error. Furthermore, we establish bounds on how well our estimator localizes the frequencies in the signal, showing that the localization error tends to zero as the number of samples grows. We verify our theoretical results in an array of numerical experiments, demonstrating that the semidefinite programming approach outperforms three classical spectral estimation techniques. [ABSTRACT FROM PUBLISHER]