Search

In the "Bits Through Queues" paper, it was hypothesized that full feedback always increases the capacity of first-in-first-out queues, except when the service time distribution is memoryless. More recently, a non-explicit sufficient condition under which feedback increases capacity was provided, along with simple examples of service times meeting this condition. While this condition yields examples where feedback is beneficial, it does not offer explicit structural properties of such service times. In this paper, we show that full feedback increases capacity whenever the service time has bounded support. This is achieved by investigating a generalized notion of feedback, with full feedback and weak feedback as particular cases., Comment: 11 pages; two-column

It was recently shown that the lossless compression of a single source $X^n$ is achievable with a notion of strong locality; any $X_i$ can be decoded from a constant number of compressed bits, with a vanishing in $n$ probability of error. By contrast, we show that for two separately encoded sources $(X^n,Y^n)$, lossless compression and strong locality is generally not possible. Specifically, we show that for the class of ``confusable'' sources, strong locality cannot be achieved whenever one of the sources is compressed below its entropy. Irrespective of $n$, for some index $i$ the probability of error of decoding $(X_i,Y_i)$ is lower bounded by $2^{-O(d)}$, where $d$ denotes the number of compressed bits accessed by the local decoder. Conversely, if the source is not confusable, strong locality is possible even if one of the sources is compressed below its entropy. Results extend to an arbitrary number of sources., Comment: 11 pages

Channel sensing consists of probing the channel from time to time to check whether or not it is active - say, because of an incoming message. When communication is sparse with information being sent once in a long while, channel sensing becomes a significant source of energy consumption. How to reliably detect messages while minimizing the receiver energy consumption? This paper addresses this problem through a reconfigurable scheme, referred to as AdaSense, which exploits the dependency between the receiver noise figure (i.e., the receiver added noise) and the receiver power consumption; a higher power typically translates into less noisy channel observations. AdaSense begins in a low power low reliability mode and makes a first tentative decision based on a few channel observations. If a message is declared, it switches to a high power high reliability mode to confirm the decision, else it sleeps for the entire duration of the second phase. Compared to prominent detection schemes such as the BMAC protocol, AdaSense provides relative energy gains that grow unbounded in the small probability of false-alarm regime, as communication gets sparser. In the non-asymptotic regime energy gains are 30% to 75% for communication scenarios typically found in the context of wake-up receivers.

Consider the compressed sensing setup where the support $s^*$ of an $m$-sparse $d$-dimensional signal $x$ is to be recovered from $n$ linear measurements with a given algorithm. Suppose that the measurements are such that the algorithm does not guarantee perfect support recovery and that true features may be missed. Can they efficiently be retrieved? This paper addresses this question through a simple error-correction module referred to as LiRE. LiRE takes as input an estimate $s_{in}$ of the true support $s^*$, and outputs a refined support estimate $s_{out}$. In the noiseless measurement setup, sufficient conditions are established under which LiRE is guaranteed to recover the entire support, that is $s_{out}$ contains $s^*$. These conditions imply, for instance, that in the high-dimensional regime LiRE can correct a sublinear in $m$ number of errors made by Orthogonal Matching Pursuit (OMP). The computational complexity of LiRE is $O(mnd)$. Experimental results with random Gaussian design matrices show that LiRE substantially reduces the number of measurements needed for perfect support recovery via Compressive Sampling Matching Pursuit, Basis Pursuit (BP), and OMP. Interestingly, adding LiRE to OMP yields a support recovery procedure that is more accurate and significantly faster than BP. This observation carries over in the noisy measurement setup. Finally, as a standalone support recovery algorithm with a random initialization, experiments show that LiRE's reconstruction performance lies between OMP and BP. These results suggest that LiRE may be used generically, on top of any suboptimal baseline support recovery algorithm, to improve support recovery or to operate with a smaller number of measurements, at the cost of a relatively small computational overhead. Alternatively, LiRE may be used as a standalone support recovery algorithm that is competitive with respect to OMP., Comment: Submitted for publication

This paper investigates the classical modulo two sum problem in source coding, but with a common observation: a transmitter observes $(X,Z)$, the other transmitter observes $(Y,Z)$, and the receiver wants to compute $X \oplus Y$ without error. Through a coupling argument, this paper establishes a new lower bound on the sum-rate when $X-Z-Y$ forms a Markov chain., Comment: Accepted for presentation at IEEE ITW 2020

How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions., Comment: Longer version of a paper accepted for presentation at the ITW 2020

This paper addresses the problem of data compression with local decoding and local update. A compression scheme has worst-case local decoding $d_{wc}$ if any bit of the raw file can be recovered by probing at most $d_{wc}$ bits of the compressed sequence, and has update efficiency of $u_{wc}$ if a single bit of the raw file can be updated by modifying at most $u_{wc}$ bits of the compressed sequence. This article provides an entropy-achieving compression scheme for memoryless sources that simultaneously achieves $ O(\log\log n) $ local decoding and update efficiency. Key to this achievability result is a novel succinct data structure for sparse sequences which allows efficient local decoding and local update. Under general assumptions on the local decoder and update algorithms, a converse result shows that $d_{wc}$ and $u_{wc}$ must grow as $ \Omega(\log\log n) $.

This paper investigates data compression that simultaneously allows local decoding and local update. The main result is a universal compression scheme for memoryless sources with the following features. The rate can be made arbitrarily close to the entropy of the underlying source, contiguous fragments of the source can be recovered or updated by probing or modifying a number of codeword bits that is on average linear in the size of the fragment, and the overall encoding and decoding complexity is quasilinear in the blocklength of the source. In particular, the local decoding or update of a single message symbol can be performed by probing or modifying a constant number of codeword bits. This latter part improves over previous best known results for which local decodability or update efficiency grows logarithmically with blocklength., Comment: 19 pages, 4 figures. v2: updated references

Immunotherapy has recently shown important clinical successes in a substantial number of oncology indications. Additionally, the tumor somatic mutation load has been shown to associate with response to these therapeutic agents, and specific mutational signatures are hypothesized to improve this association, including signatures related to pathogen insults. We sought to study in silico the validity of these observations and how they relate to each other. We first addressed whether somatic mutations typically involved in cancer may increase, in a statistically meaningful manner, the similarity between common pathogens and the human exome. Our study shows that common mutagenic processes increase, in the upper range of biologically plausible frequencies, the similarity between cancer exomes and pathogen DNA at a scale of 12-16 nucleotide sequences and established that this increased similarity is due to the specific mutation distribution of the considered mutagenic processes. Next, we studied the impact of mutation rate and showed that increasing mutation rate generally results in an increased similarity between the cancer exome and pathogen DNA, at a scale of 4-5 amino acids. Finally, we investigated whether the considered mutational processes result in amino-acid changes with functional relevance that are more likely to be immunogenic. We showed that functional tolerance to mutagenic processes across species generally suggests more resilience to mutagenic processes that are due to exposure to elements of nature than to mutagenic processes that are due to exposure to cancer-causing artificial substances. These results support the idea that recognition of pathogen sequences as well as differential functional tolerance to mutagenic processes may play an important role in the immune recognition process involved in tumor infiltration by lymphocytes.

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.

The capacity under strong asynchronism was recently shown to be essentially unaffected by the imposed output sampling rate $\rho$ and decoding delay $d$---the elapsed time between when information is available at the transmitter and when it is decoded. This paper examines this result in the finite blocklength regime and shows that, by contrast with capacity, the second order term in the rate expansion is sensitive to both parameters. When the receiver must exactly locate the sent codeword, that is $d=n$ where $n$ denotes blocklength, the second order term in the rate expansion is of order $\Theta(1/\rho)$ for any $\rho=O(1/\sqrt{n})$---and $\rho =\omega(1/n)$ for otherwise reliable communication is impossible. However, if $\rho=\omega(1/\sqrt{n})$ then the second order term is the same as under full sampling and is given by a standard $O(\sqrt{n})$ term whose dispersion constant only depends on the level of asynchronism. This second order term also corresponds to the case of the slightly relaxed delay constraint $d\leq n(1+o(1))$ for any $\rho=\omega(1/n)$.

In their $1996$ paper Anantharam and Verd\'u showed that feedback does not increase the capacity of a queue when the service time is exponentially distributed. Whether this conclusion holds for general service times has remained an open question which this paper addresses. Two main results are established for both the discrete-time and the continuous-time models. First, a sufficient condition on the service distribution for feedback to increase capacity under FIFO service policy. Underlying this condition is a notion of weak feedback wherein instead of the queue departure times the transmitter is informed about the instants when packets start to be served. Second, a condition in terms of output entropy rate under which feedback does not increase capacity. This condition is general in that it depends on the output entropy rate of the queue but explicitly depends neither on the queue policy nor on the service time distribution. This condition is satisfied, for instance, by queues with LCFS service policies and bounded service times.

The per-sample zero-dispersion channel model of the optical fiber is considered. It is shown that capacity is uniquely achieved by an input probability distribution that has continuous uniform phase and discrete amplitude that takes on finitely many values. This result holds when the channel is subject to general input cost constraints, that include a peak amplitude constraint and a joint average and peak amplitude constraint., Comment: 18 pages; Submitted for review to the IEEE Transactions on Information Theory

The minimum energy, and, more generally, the minimum cost, to transmit one bit of information has been recently derived for bursty communication when information is available infrequently at random times at the transmitter. Furthermore, it has been shown that even if the receiver is constrained to sample only a fraction $\rho\in (0,1]$ of the channel outputs, there is no capacity penalty. That is, for any strictly positive sampling rate $\rho>0$, the asynchronous capacity per unit cost is the same as under full sampling, i.e., when $\rho=1$. Moreover, there is no penalty in terms of decoding delay. The above results are asymptotic in nature, considering the limit as the number $B$ of bits to be transmitted tends to infinity, while the sampling rate $\rho$ remains fixed. A natural question is then whether the sampling rate $\rho(B)$ can drop to zero without introducing a capacity (or delay) penalty compared to full sampling. We answer this question affirmatively. The main result of this paper is an essentially tight characterization of the minimum sampling rate. We show that any sampling rate that grows at least as fast as $\omega(1/B)$ is achievable, while any sampling rate smaller than $o(1/B)$ yields unreliable communication. The key ingredient in our improved achievability result is a new, multi-phase adaptive sampling scheme for locating transient changes, which we believe may be of independent interest for certain change-point detection problems., Comment: Accepted for publication in the IEEE Transactions on Information Theory

The construction of deletion codes for the Levenshtein metric is reduced to the construction of codes over the integers for the Manhattan metric by run length coding. The latter codes are constructed by expurgation of translates of lattices. These lattices, in turn, are obtained from Construction~A applied to binary codes and $\Z_4-$codes. A lower bound on the size of our codes for the Manhattan distance are obtained through generalized theta series of the corresponding lattices., Comment: 2 figs; presented in part in ISIT 2013; submitted to IEEE trans. on Information Theory

This paper establishes the rate region for a class of source coding function computation setups where sources of information are available at the nodes of a tree and where a function of these sources must be computed at the root. The rate region holds for any function as long as the sources' joint distribution satisfies a certain Markov criterion. This criterion is met, in particular, when the sources are independent. This result recovers the rate regions of several function computation setups. These include the point-to-point communication setting with arbitrary sources, the noiseless multiple access network with "conditionally independent sources," and the cascade network with Markovian sources., Comment: 36 pages, Submitted to IEEE Transactions on Information Theory

A receiver wants to compute a function of two correlated sources separately observed by two transmitters. One of the transmitters may send a possibly private message to the other transmitter in a cooperation phase before both transmitters communicate to the receiver. For this network configuration this paper investigates both a function computation setup, wherein the receiver wants to compute a given function of the sources exactly, and a rate distortion setup, wherein the receiver wants to compute a given function within some distortion. For the function computation setup, a general inner bound to the rate region is established and shown to be tight in a number of cases: partially invertible functions, full cooperation between transmitters, one-round point-to-point communication, two-round point-to-point communication, and the cascade setup where the transmitters and the receiver are aligned. In particular it is shown that the ratio of the total number of transmitted bits without cooperation and the total number of transmitted bits with cooperation can be arbitrarily large. Furthermore, one bit of cooperation suffices to arbitrarily reduce the amount of information both transmitters need to convey to the receiver. For the rate distortion version, an inner bound to the rate region is exhibited which always includes, and sometimes strictly, the convex hull of Kaspi-Berger's related inner bounds. The strict inclusion is shown via two examples., Comment: 31 pages, Submitted to IEEE Transactions on Information Theory

The minimum energy, and, more generally, the minimum cost, to transmit one bit of information has been recently derived for bursty communication when information is available infrequently at random times at the transmitter. This result assumes that the receiver is always in the listening mode and samples all channel outputs until it makes a decision. If the receiver is constrained to sample only a fraction f>0 of the channel outputs, what is the cost penalty due to sparse output sampling? Remarkably, there is no penalty: regardless of f>0 the asynchronous capacity per unit cost is the same as under full sampling, ie, when f=1. There is not even a penalty in terms of decoding delay---the elapsed time between when information is available until when it is decoded. This latter result relies on the possibility to sample adaptively; the next sample can be chosen as a function of past samples. Under non-adaptive sampling, it is possible to achieve the full sampling asynchronous capacity per unit cost, but the decoding delay gets multiplied by 1/f. Therefore adaptive sampling strategies are of particular interest in the very sparse sampling regime., Comment: Submitted to the IEEE Transactions on Information Theory

The one-bit deletion and duplication channel is investigated. An input to this channel consists of a block of bits which experiences either a deletion, or a duplication, or remains unchanged. For this channel a capacity expression is obtained in a certain asymptotic regime where the deletion and duplication probabilities tend to zero. As a corollary, we obtain an asymptotic expression for the capacity of the segmented deletion and duplication channel where the input now consists of several blocks and each block independently experiences either a deletion, or a duplication, or remains unchanged., Comment: Allerton 2012

A random walk (or a Wiener process), possibly with drift, is observed in a noisy or delayed fashion. The problem considered in this paper is to estimate the first time \tau the random walk reaches a given level. Specifically, the p-moment (p\geq 1) optimization problem \inf_\eta \ex|\eta-\tau|^p is investigated where the infimum is taken over the set of stopping times that are defined on the observation process. When there is no drift, optimal stopping rules are characterized for both types of observations. When there is a drift, upper and lower bounds on \inf_\eta \ex|\eta-\tau|^p are established for both types of observations. The bounds are tight in the large-level regime for noisy observations and in the large-level-large-delay regime for delayed observations. Noteworthy, for noisy observations there exists an asymptotically optimal stopping rule that is a function of a single observation. Simulation results are provided that corroborate the validity of the results for non-asymptotic settings., Comment: To appear in the IEEE Transactions on Information Theory

A receiver wants to compute a function f of two correlated sources X and Y and side information Z. What is the minimum number of bits that needs to be communicated by each transmitter? In this paper, we derive inner and outer bounds to the rate region of this problem which coincide in the cases where f is partially invertible and where the sources are independent given the side information. These rate regions point to an important difference with the single source case. Whereas for the latter it is sufficient to consider independent sets of some suitable characteristic graph, for multiple sources such a restriction is suboptimal and multisets are necessary., Comment: 11 pages, Submitted to IEEE Transactions on Information Theory

Several aspects of the problem of asynchronous point-to-point communication without feedback are developed when the source is highly intermittent. In the system model of interest, the codeword is transmitted at a random time within a prescribed window whose length corresponds to the level of asynchronism between the transmitter and the receiver. The decoder operates sequentially and communication rate is defined as the ratio between the message size and the elapsed time between when transmission commences and when the decoder makes a decision. For such systems, general upper and lower bounds on capacity as a function of the level of asynchronism are established, and are shown to coincide in some nontrivial cases. From these bounds, several properties of this asynchronous capacity are derived. In addition, the performance of training-based schemes is investigated. It is shown that such schemes, which implement synchronization and information transmission on separate degrees of freedom in the encoding, cannot achieve the asynchronous capacity in general, and that the penalty is particularly significant in the high-rate regime., Comment: 27 pages, 8 figures, submitted to the IEEE Transactions on Information Theory

The maximum size of a binary code is studied as a function of its length N, minimum distance D, and minimum codeword weight W. This function B(N,D,W) is first characterized in terms of its exponential growth rate in the limit as N tends to infinity for fixed d=D/N and w=W/N. The exponential growth rate of B(N,D,W) is shown to be equal to the exponential growth rate of A(N,D) for w <= 1/2, and equal to the exponential growth rate of A(N,D,W) for 1/2< w <= 1. Second, analytic and numerical upper bounds on B(N,D,W) are derived using the semidefinite programming (SDP) method. These bounds yield a non-asymptotic improvement of the second Johnson bound and are tight for certain values of the parameters.

The capacity per unit cost, or equivalently minimum cost to transmit one bit, is a well-studied quantity. It has been studied under the assumption of full synchrony between the transmitter and the receiver. In many applications, such as sensor networks, transmissions are very bursty, with small amounts of bits arriving infrequently at random times. In such scenarios, the cost of acquiring synchronization is significant and one is interested in the fundamental limits on communication without assuming a priori synchronization. In this paper, we show that the minimum cost to transmit B bits of information asynchronously is (B + \bar{H})k_sync, where k_sync is the synchronous minimum cost per bit and \bar{H} is a measure of timing uncertainty equal to the entropy for most reasonable arrival time distributions. This result holds when the transmitter can stay idle at no cost and is a particular case of a general result which holds for arbitrary cost functions., Comment: to be published in the IEEE Transactions on Information Theory

Given a Gaussian random walk (or a Wiener process), possibly with drift, observed through noise, we consider the problem of estimating its first-passage time $\tau_\ell$ of a given level $\ell$ with a stopping time $\eta$ defined over the noisy observation process. Main results are upper and lower bounds on the minimum mean absolute deviation $\inf_\eta \ex|\eta-\tau_\ell|$ which become tight as $\ell\to\infty$. Interestingly, in this regime the estimation error does not get smaller if we allow $ \eta$ to be an arbitrary function of the entire observation process, not necessarily a stopping time. In the particular case where there is no drift, we show that it is impossible to track $\tau_\ell$: $\inf_\eta \ex|\eta-\tau_\ell|^p=\infty$ for any $\ell>0$ and $p\geq1/2$., Comment: Reprint of the original article published in the Annals of Applied Probability

We consider asynchronous point-to-point communication. Building on a recently developed model, we show that training based schemes, i.e., communication strategies that separate synchronization from information transmission, perform suboptimally at high rate., Comment: To appear in the proceedings of the 2009 IEEE Information Theory Workshop (Taormina)

Relay attacks are a major concern for RFID systems: during an authentication process an adversary transparently relays messages between a verifier and a remote legitimate prover. We present an authentication protocol suited for RFID systems. Our solution is the first that prevents relay attacks without degrading the authentication security level: it minimizes the probability that the verifier accepts a fake proof of identity, whether or not a relay attack occurs., Comment: 10 pages

We consider the `one-shot frame synchronization problem' where a decoder wants to locate a sync pattern at the output of a channel on the basis of sequential observations. We assume that the sync pattern of length N starts being emitted at a random time within some interval of size A, that characterizes the asynchronism level between the transmitter and the receiver. We show that a sequential decoder can optimally locate the sync pattern, i.e., exactly, without delay, and with probability approaching one as N tends to infinity, if and only if the asynchronism level grows as O(exp(N*k)), with k below the `synchronization threshold,' a constant that admits a simple expression depending on the channel. This constant is the same as the one that characterizes the limit for reliable asynchronous communication, as was recently reported by the authors. If k exceeds the synchronization threshold, any decoder, sequential or non-sequential, locates the sync pattern with an error that tends to one as N tends to infinity. Hence, a sequential decoder can locate a sync pattern as well as the (non-sequential) maximum likelihood decoder that operates on the basis of output sequences of maximum length A+N-1, but with much fewer observations., Comment: 6 pages

We consider asynchronous communication over point-to-point discrete memoryless channels. The transmitter starts sending one block codeword at an instant that is uniformly distributed within a certain time period, which represents the level of asynchronism. The receiver, by means of a sequential decoder, must isolate the message without knowing when the codeword transmission starts but being cognizant of the asynchronism level A. We are interested in how quickly can the receiver isolate the sent message, particularly in the regime where A is exponentially larger than the codeword length N, which we refer to as `strong asynchronism.' This model of sparse communication may represent the situation of a sensor that remains idle most of the time and, only occasionally, transmits information to a remote base station which needs to quickly take action. The first result shows that vanishing error probability can be guaranteed as N tends to infinity while A grows as Exp(N*k) if and only if k does not exceed the `synchronization threshold,' a constant that admits a simple closed form expression, and is at least as large as the capacity of the synchronized channel. The second result is the characterization of a set of achievable strictly positive rates in the regime where A is exponential in N, and where the rate is defined with respect to the expected delay between the time information starts being emitted until the time the receiver makes a decision. As an application of the first result we consider antipodal signaling over a Gaussian channel and derive a simple necessary condition between A, N, and SNR for achieving reliable communication., Comment: 26 pages

Wyner's wiretap channel is extended to parallel broadcast channels and fading channels with multiple receivers. In the first part of the paper, we consider the setup of parallel broadcast channels with one sender, multiple intended receivers, and one eavesdropper. We study the situations where the sender broadcasts either a common message or independent messages to the intended receivers. We derive upper and lower bounds on the common-message-secrecy capacity, which coincide when the users are reversely degraded. For the case of independent messages we establish the secrecy sum-capacity when the users are reversely degraded. In the second part of the paper we apply our results to fading channels: perfect channel state information of all intended receivers is known globally, whereas the eavesdropper channel is known only to her. For the common message case, a somewhat surprising result is proven: a positive rate can be achieved independently of the number of intended receivers. For independent messages, an opportunistic transmission scheme is presented that achieves the secrecy sum-capacity in the limit of large number of receivers. Our results are stated for a fast fading channel model. Extensions to the block fading model are also discussed., Comment: Submitted to IEEE Transactions on Information Theory, Special issue on Information Theoretic Security

A novel quickest detection setting is proposed which is a generalization of the well-known Bayesian change-point detection model. Suppose \{(X_i,Y_i)\}_{i\geq 1} is a sequence of pairs of random variables, and that S is a stopping time with respect to \{X_i\}_{i\geq 1}. The problem is to find a stopping time T with respect to \{Y_i\}_{i\geq 1} that optimally tracks S, in the sense that T minimizes the expected reaction delay E(T-S)^+, while keeping the false-alarm probability P(T