Search

Shannon’s analysis of the fundamental capacity limits for memoryless communication channels has been refined over time. In this paper, the maximum volume M_ \mathrm avg^*(n,\epsilon ) of length- n codes subject to an average decoding error probability \epsilon is shown to satisfy the following tight asymptotic lower and upper bounds as n \to \infty : , where C is the Shannon capacity, V_\epsilon is the \epsilon -channel dispersion, or second-order coding rate, Q and \overline A_\epsilon are explicitly identified. This expression holds under mild regularity assumptions on the channel, including nonsingularity. The gap \overline A_\epsilon - \underline A_\epsilon is one nat for weakly symmetric channels in the Cover–Thomas sense, and typically a few nats for other symmetric channels, for the binary symmetric channel, and for the $Z$ channel. The derivation is based on strong large-deviations analysis and refined central limit asymptotics. A random coding scheme that achieves the lower bound is presented. The codewords are drawn from a capacity-achieving input distribution modified by an O(1/\sqrt {n}) correction term. [ABSTRACT FROM PUBLISHER]

In this paper, we investigate order-preserving sparse coding for classifying structured data whose atomic features possess ordering relationships. Examples include time sequences where individual frame-wise features are temporally ordered, as well as still images (landscape, street view, etc.) where different regions of the image are spatially ordered. Classification of these structured data is often tackled by first decomposing the input data into individual atomic features, then performing sparse coding or other processing for each atomic feature vector independently, and finally aggregating individual responses to classify the input data. However, this heuristic approach ignores the underlying order of the individual atomic features within the input data, and results in suboptimal discriminative capability. In this work, we introduce an order preserving regularizer which aims to preserve the ordering structure of the reconstruction coefficients within the sparse coding framework. An efficient Nesterov-type smooth approximation method is developed for optimization of the new regularization criterion, with theoretically guaranteed error bound. We perform extensive experiments for time series classification on a synthetic dataset, several machine learning benchmarks, and an RGB-D human activity dataset. We also report experiments for scene classification on a benchmark image dataset. The encoded representation is discriminative and robust, and our classifier outperforms state-of-the-art methods on these tasks. [ABSTRACT FROM PUBLISHER]

When information is to be transmitted over a unknown, possibly unreliable channel, an erasure option at the decoder is desirable. Using constant-composition random codes, we propose a generalization of Csiszár and Körner's maximum mutual information (MMI) decoder with an erasure option for discrete memoryless channels. The new decoder is parameterized by a weighting function that is designed to optimize the fundamental tradeoff between undetected-error and erasure exponent for a compound class of channels. The class of weighting functions may be further enlarged to optimize a similar tradeoff for list decoders--in that case, undetected-error probability is replaced with average number of incorrect messages in the fist. Explicit solutions are identified. The optimal exponents admit simple expressions in terms of the sphere-packing exponent, at all rates below capacity. For small erasure exponents, these expressions coincide with those derived by Forney (1968) for symmetric channels, using maximum a posteriori decoding. Thus, for those channels at least, ignorance of the channel law is inconsequential. Conditions for optimality of the Csiszfir-Körner rule and of the simpler empirical-mutual-information thresholding rule are identified. The error exponents are evaluated numerically for the binary symmetric channel. [ABSTRACT FROM AUTHOR]

An analysis of steganographic systems subject to the following perfect undetectability condition is presented in this paper. Following embedding of the message into the covertext, the resulting stegotext is required to have exactly the same probability distribution as the covertext. Then no statistical test can reliably detect the presence of the hidden message. We refer to such steganographic schemes as perfectly secure. A few such schemes have been proposed in recent literature, but they have vanishing rate. We prove that communication performance can potentially be vastly improved; specifically, our basic setup assumes independent and identically distributed (i.i.d.) covertext, and we construct perfectly secure steganographic codes from public watermarking codes using binning methods and randomized permutations of the code. The permutation is a secret key shared between encoder and decoder. We derive (positive) capacity and random-coding exponents for perfectly secure steganographic systems. The error exponents provide estimates of the code length required to achieve a target low error probability. In some applications, steganographic communication may be disrupted by an active warden, modeled here by a compound discrete memoryless channel (DMC). The transmitter and warden are subject to distortion constraints. We address the potential loss in communication performance due to the perfect-security requirement. This loss is the same as the loss obtained under a weaker order-1 steganographic requirement that would just require matching of first-order marginals of the covertext and stegotext distributions. Furthermore, no loss occurs if the covertext distribution is uniform and the distortion metric is cyclically symmetric; steganographic capacity is then achieved by randomized linear codes. Our framework may also be useful for developing computationally secure steganographic systems that have near-optimal communication performance. [ABSTRACT FROM AUTHOR]

Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning. [ABSTRACT FROM AUTHOR]

This paper derives fundamental performance bounds for statistical estimation of parametric surfaces embedded in R3. Unlike conventional pixel-based image reconstruction approaches, our problem is reconstruction of the shape of binary or homogeneous objects. The fundamental uncertainty of such estimation problems can be represented by global confidence regions, which facilitate geometric inference and optimization of the imaging system. Compared to our previous work on global confidence region analysis for curves [two-dimensional (2-D) shapes], computation of the probability that the entire surface estimate lies within the confidence region is more challenging because a surface estimate is an inhomogeneous random field continuously indexed by a 2-D variable. We derive an asymptotic lower bound to this probability by relating it to the exceedence probability of a higher dimensional Gaussian random field, which can, in turn, be evaluated using the tube formula due to Sun. Simulation results demonstrate the tightness of the resulting bound and the usefulness of the three-dimensional global confidence region approach. [ABSTRACT FROM AUTHOR]

The Karhunen-Loéve transform (KLT) is known to be optimal for high-rate transform coding of Gaussian vectors for both fixed-rate and variable-rate encoding. The KLT is also known to be suboptimal for some non-Gaussian models. This paper proves high-rate optimality of the KLT for variable-rate encoding of a broad class of non-Gaussian vectors: Gaussian vector-scale mixtures (GVSM), which extend the Gaussian scale mixture (GSM) model of natural signals. A key concavity property of the scalar GSM (same as the scalar GVSM) is derived to complete the proof. Optimality holds under a broad class of quadratic criteria, which include mean-squared error (MSE) as well as generalized f-divergence loss in estimation and binary classification systems. Finally, the theory is illustrated using two applications: signal estimation in multiplicative noise and joint optimization of classification/reconstruction systems. [ABSTRACT FROM AUTHOR]

Modulo lattice additive noise (MLAN) channels appear in the analysis of structured binning codes for Costa's dirty-paper channel and of nested lattice codes for the additive white Gaussian noise (AWGN) channel. In this paper, we derive a new lower bound on the error exponents of the MLAN channel. With a proper choice of the shaping lattice and the scaling parameter, the new lower bound coincides with the random-coding lower bound on the error exponents of the AWGN channel at the same signal-to-noise ratio (SNR) in the sphere-packing and straight-line regions. This result implies that, at least for rates close to channel capacity, 1) writing on dirty paper is as reliable as writing on clean paper; and 2) lattice encoding and decoding suffer no loss of error exponents relative to the optimal codes (with maximum-likelihood decoding) for the AWGN channel. [ABSTRACT FROM AUTHOR]

A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on the minimum cross-entropy distribution or apply only to distributions with a bounded-volume support or address only equality constraints. The results of this work hold for general moment inequality constraints for probability distributions with possibly unbounded support, and the technical conditions are explicitly on the underlying generalized moment functions. An analytical characterization of the maxent distribution is also derived using results from the theory of constrained optimization in infinite-dimensional norms linear spaces. Several auxiliary results of independent Interest pertaining to certain properties of convex coercive functions are also presented. [ABSTRACT FROM AUTHOR]

This paper develops a joint hashing/watermarking scheme in which a short hash of the host signal is available to a detector. Potential applications include content tracking on public networks and forensic identification. The host data into which the watermark is embedded are selected from a secret subset of the full-frame discrete cosine transform of an image, and the watermark is inserted through multiplicative embedding. The hash is a binary version of selected original image coefficients. We propose a maximum likelihood watermark detector based on a statistical image model. The availability of a hash as side information to the detector modifies the posterior distribution of the marked coefficients. We derive Chernoff bounds on the receiver operating characteristic performance of the detector. We show that host-signal interference can be rejected if the hash function is suitably designed. The relative difficulty of an eavesdropper's detection problem is also determined; the eavesdropper does not know the secret key used. Monte Carlo simulations are performed using photographic test images. Finally, various attacks on the watermarked image are introduced to study the robustness of the derived detectors. The joint hashing/watermarking scheme outperforms the traditional "hashless" watermarking technique. [ABSTRACT FROM AUTHOR]

Rates of reliable transmission of hidden information are derived for watermarking problems involving parallel Gaussian sources, which are often used to model host images and audio signals. Constraints are imposed on the average squared-error distortion that can be introduced by the information hider and by the attacker. When distortions are measured with respect to the original host data, the optimal covert and attack channels are two banks of Gaussian test channels. The solution to the watermarking game involves an optimal allocation of distortions by the information hider and by the attacker to the different channels. A fast algorithm is given for computing the optimal solution based on duality theory. For each channel, we derive analytical expressions for two asymptotic regimes: weak and strong host signals. Finally, we extend these results to the class of stationary Gaussian host signals with bounded, continuous spectral density. The analysis also provides an upper bound on watermarking capacity for non-Gaussian host signals. [ABSTRACT FROM AUTHOR]

This paper develops a game-theoretic methodology to design and embed M messages in signals and images in the presence of an adversary. Here, M is assumed to be subexponential in the signal's sample size (zero-rate transmission), and the embedding is done using spread-spectrum watermarking. The detector performs statistical hypothesis testing. The system is designed to minimize probability of error under the worst-case attack in a prescribed class of attacks. The variables in this game are probability distributions for the watermarker and attacker. Analytical solutions are derived under the assumption of Gaussian host vectors, watermarks and attacks, and squared-error distortion constraints for the watermarker and the attacker. The Karhunen-Lo&egamma;ve transform (KLT) plays a central role in this study. The optimal distributions for the watermarker and the attacker are Gaussian test channels applied to the KLT coefficients; the game is then reduced to a maxmin power-allocation problem between the channels. As a byproduct of this analysis, we can determine the optimal tradeoff between using the most efficient (in terms of detection performance) signal components for transmission and spreading the transmission across many components (to fool the attacker's attempts to eliminate the watermark). We also conclude that in this framework, additive watermarks are suboptimal; they are, however, nearly optimal in a small-distortion regime. The theory is applied to watermarking of autoregressive processes and to wavelet-based image watermarking. The optimal watermark design outperforms conventional designs based on heuristic power allocations and/or simple correlation detectors. [ABSTRACT FROM AUTHOR]

An information-theoretic analysis of information hiding is presented in this paper, forming the theoretical basis for design of information-hiding systems. Information hiding is an emerging research area which encompasses applications such as copyright protection for digital media, watermarking, fingerprinting, steganography, and data embedding. In these applications, information is hidden within a host data set and is to be reliably communicated to a receiver. The host data set is intentionally corrupted, but in a covert way, designed to be imperceptible to a casual analysis. Next, an attacker may seek to destroy this hidden information, and for this purpose, introduce additional distortion to the data set. Side information (in the form of cryptographic keys and/or information about the host signal) may be available to the information hider and to the decoder. We formalize these notions and evaluate the hiding capacity, which upper-bounds the rates of reliable transmission and quantifies the fundamental tradeoff between three quantities: the achievable information-hiding rates and the allowed distortion levels for the information hider and the attacker. The hiding capacity is the value of a game between the information hider and the attacker. The optimal attack strategy is the solution of a particular rate-distortion problem, and the optimal hiding strategy is the solution to a channel-coding problem. The hiding capacity is derived by extending the Gel'fand-Pinsker theory of communication with side information at the encoder. The extensions include the presence of distortion constraints, side information at the decoder, and unknown communication channel. Explicit formulas for capacity are given in several cases, including Bernoulli and Gaussian problems, as well as the important special case of small distortions. In some cases, including the last two above, the hiding capacity is the same whether or not the decoder knows the host data set. It is shown that... [ABSTRACT FROM AUTHOR]

In this papers we establish an equivalence between two conceptually different methods of signal estimation under modeling uncertainty viz. set-theoretic (ST) estimation and maximum entropy (maxent) MAP estimation. The first method assumes constraints on the signal to be estimated, and the second assumes constraints on a probability distribution for the signal. We provide broad conditions under which the two aforementioned estimation paradigms produce the same signal estimate. We also show how the maxent formalism can be used to provide solutions to three important problems: How to select sizes of constraint sets in ST estimation (the analysis highlights the role of shrinkage); how to choose the values of parameters in regularized restoration when using multiple regularization functionals; and how to trade off model complexity and goodness of fit in a model selection problem. [ABSTRACT FROM AUTHOR]

Details the derivation of information-theoretic bounds on the performance statistical object recognition systems. Geometric representation of probability sets and information distances; Performance bounds for tank imagery.

Evaluates the data-hiding capacity of image sources using an information-theoretic model. Construction of a hyper media document; Rejection of the host-signal interference at the decoder; Derivation of the fundamental limits on watermarking.

Presents an information-theoretic analysis of statistical dependencies between image wavelet coefficients. Formulation of the modeling problem in terms of mutual information; Illustration of concepts in the case of an AR-1 Gaussian image process; Information on reduced-order models.

Presents a study which examined an approach to image denoising based on complexity regularization. Overview of complexity-regularized denoising algorithms; Theoretical analysis; Experimental results.

Investigates the design of subband coders without the traditional perfect-reconstruction constraint on the filters. Subband coder model; Design criterion; Fundamental properties of optimal subband coder; Optimal filters for a given bit allocation.

Introduces a method that simultaneously preserves the benefits of higher-order interpolative schemes such as control grid interpolation (CGI). Block matching and CGI algorithms; Adaptive version of overlapped block motion compensation; Video-coding results comparing smoothness-adapted versus nonadapted motion-compensated prediction methods.

Presents a study which applied the complexity-regularization principle to statistical ill-posed inverse problems in imaging. Methodology; Results; Discussion.

Presents information on a study which analyzed the rate-distortion-optimal constrained filterbanks and its application to infinite impulse response (IIR) and finite IR biorthogonal designs. Properties of the filter banks; Conclusions.

Investigates various connections between shrinkage methods and Maximum A Posteriori estimation using Gaussian and complexity priors. Application of the Minimum Description Length principle; Development of analytical expressions implied by Gaussian distributions.

Presents information on a study that explored the application of signal-adapted filter banks to image coding. Information on the proposed coding paradigm to optimize rate-distortion (R/D) performance; Design of the codebook; Description of problem solving using a fast optimization algorithm; Conclusion.

Studies the design of signal-adapted far-infrared radiation paraunitary filter banks, using energy compaction as the adaptation criterion. Information on the semi-infinite problem (SIP); Discussion of the SIP algorithms; Numerical results of the SIP.

Considers the problem of designing a perfect reconstruction, quadrature-mirror filter (QMF) bank adapted to input signal statistics. Views on maximization of the coding gain; Theoretical analysis of the linear semi-infinite programming (SIP) problem; Application of discretization and cutting plane algorithms.

Presents a systematic approach to forward-motion-compensated predictive video coding. Definition of a flexible model which represents motion fields; How inhomogeneity and spatial coherence properties of motion fields are captured.

The article discusses the IEEE Transactions on Information Forensics and Security (T-IFS) supplement featured within the publication.

This article presents information on the October 2004 issue of the IEEE Transactions on Signal Processing. This issue is the first among three supplements on secure media. It contains ten papers which covers a broad range of topics and were selected based on their high level of technical quality, mathematical rigor, significance and novelty. The topics include steganography, watermarking codes, temporal geometric robustness of video watermarks, cryptography for multimedia, performance analysis of biometric systems.

This paper presents a new approach to image-set-based face recognition, where each training and testing example is a set of face images captured from varying poses, illuminations, expressions, and resolutions. While a number of image set based face recognition methods have been proposed in recent years, most of them model each face image set as a single linear subspace or as the union of linear subspaces, which may lose some discriminative information for face image set representation. To address this shortcoming, we propose exploiting statistics information as feature representations for face image sets and develop a localized multikernel metric learning algorithm to effectively combine different statistics for recognition. Moreover, we propose a localized multikernel multimetric learning method to jointly learn multiple feature-specific distance metrics in the kernel spaces, one for each statistic feature, to better exploit complementary information for recognition. Our methods achieve state-of-the-art performance on four widely used video face datasets including the Honda, MoBo, YouTube Celebrities, and YouTube Face datasets. [ABSTRACT FROM AUTHOR]

We address the problem of computing fundamental performance bounds for estimation of object boundaries from noisy measurements in inverse problems, when the boundaries are parameterized by a finite number of unknown variables. Our model applies to multiple unknown objects, each with its own unknown gray level, or color, and boundary parameterization, on an arbitrary known background. While such fundamental bounds on the performance of shape estimation algorithms can in principle be derived from the Cramér-Rao lower bounds, very few results have been reported due to the difficulty of computing the derivatives of a functional with respect to shape deformation. In this paper, we provide a general formula for computing CramérRao lower bounds in inverse problems where the observations are related to the object by a general linear transform, followed by a possibly nonlinear and noisy measurement system. As an illustration, we derive explicit formulas for computed tomography, Fourier imaging, and deconvolution problems. The bounds reveal that highly accurate parametric reconstructions are possible in these examples, using severely limited and noisy data. [ABSTRACT FROM AUTHOR]

Presents methods for computing performance limits for two-dimensional parametric shape estimation in nonlinear inverse scattering problems with an application to passive radar imaging. Analysis for nonlinear inverse scattering problems using Cramer-Rao lower bounds; Use of normalized Fourier descriptors for boundary parameterization; Extension of the global confidence region analysis using transverse electric polarized incident plane waves.

We consider the problems of motion-compensated frame interpolation (MCFI) and bidirectional prediction in a video coding environment. These applications generally require good motion estimates at the decoder. We use a multiscale optical-flow-based motion estimator that provides smooth, natural motion fields under bit-rate constraints. These motion estimates scale well with change in temporal resolution and provide considerable flexibility in the design and operation of coders and decoders. In the MCFI application, this estimator provides excellent interpolated frames that are superior to those of conventional motion estimators, both visually and in terms of peak signal-to-noise ratio (PSNR). We also consider the effect of occlusions in the bidirectional prediction application and introduce a dense label field that complements our motion estimator. This label field enables us to adaptively weight the forward and backward predictions and gives us substantial visual and PSNR improvements in the covered/uncovered regions of the sequence [ABSTRACT FROM PUBLISHER]

Let $X_{i}, i~\in V$ form a Markov random field (MRF) represented by an undirected graph $G = (V,E)$ , and $V'$ be a subset of $V$. We determine the smallest graph that can always represent the subfield $X_{i}, i~\in V'$ as an MRF. Based on this result, we obtain a necessary and sufficient condition for a subfield of a Markov tree to be also a Markov tree. When $G$ is a path so that $X_{i}, i~\in V$ form a Markov chain, it is known that the $I$ -Measure is always nonnegative (Kawabata and Yeung in 1992). We prove that Markov chain is essentially the only MRF such that the $I$ -Measure is always nonnegative. By applying our characterization of the smallest graph representation of a subfield of an MRF, we develop a recursive approach for constructing information diagrams for MRFs. Our work is built on the set-theoretic characterization of an MRF (Yeung et al. in 2002). [ABSTRACT FROM AUTHOR]