Search

The distributed representation of correlated multi-view images is an important problem that arise in vision sensor networks. This paper concentrates on the joint reconstruction problem where the distributively compressed correlated images are jointly decoded in order to improve the reconstruction quality of all the compressed images. We consider a scenario where the images captured at different viewpoints are encoded independently using common coding solutions (e.g., JPEG, H.264 intra) with a balanced rate distribution among different cameras. A central decoder first estimates the underlying correlation model from the independently compressed images which will be used for the joint signal recovery. The joint reconstruction is then cast as a constrained convex optimization problem that reconstructs total-variation (TV) smooth images that comply with the estimated correlation model. At the same time, we add constraints that force the reconstructed images to be consistent with their compressed versions. We show by experiments that the proposed joint reconstruction scheme outperforms independent reconstruction in terms of image quality, for a given target bit rate. In addition, the decoding performance of our proposed algorithm compares advantageously to state-of-the-art distributed coding schemes based on disparity learning and on the DISCOVER.

Manifold models provide low-dimensional representations that are useful for processing and analyzing data in a transformation-invariant way. In this paper, we study the problem of learning smooth pattern transformation manifolds from image sets that are observations of geometrically transformed signals. In order to construct a manifold, we build a representative pattern whose transformations accurately fit various input images. The pattern is formed by selecting a good common sparse approximation of the images with parametric and smooth atoms. We examine two aspects of the manifold building problem, where we first target an accurate transformation-invariant approximation of the input images, and then extend this solution for their classification. For the approximation problem, we propose a greedy method that constructs a representative pattern by selecting analytic atoms from a continuous dictionary manifold. We present a DC (Difference-of-Convex) optimization scheme which is applicable for a wide range of transformation and dictionary models, and demonstrate its application to transformation manifolds generated by the rotation, translation and anisotropic scaling of a reference pattern. Then, we generalize this approach to a setting with multiple transformation manifolds, where each manifold represents a different class of signals. We present an iterative multiple manifold building algorithm such that the classification accuracy is promoted in the joint selection of atoms. Experimental results suggest that the proposed methods yield high accuracy in the approximation and classification of data in comparison with some reference methods, while achieving invariance to geometric transformations due to the transformation manifold model.

We investigate the problem of distributed sensors' failure detection in networks with a small number of defective sensors, whose measurements differ significantly from neighboring sensor measurements. Defective sensors are represented by non-zero values in binary sparse signals. We build on the sparse nature of the binary sensor failure signals and propose a new distributed detection algorithm based on Group Testing (GT). The distributed GT algorithm estimates the set of defective sensors from a small number of linearly independent binary messages exchanged by the sensors. The distributed GT algorithm uses a low complexity distance decoder that is robust to noisy messages. We first consider networks with only one defective sensor and determine the minimal number of linearly independent messages needed for detection of the defective sensor with high probability. We then extend our study to the detection of multiple defective sensors by modifying appropriately the message exchange protocol and the decoding procedure. We show through experimentation that, for small and medium sized networks, the number of messages required for successful detection is actually smaller than the minimal number computed in the analysis. Simulations demonstrate that the proposed method outperforms methods based on random walk measurements collection in terms of detection performance and convergence rate. Finally, the proposed method is resilient to network dynamics due to the effective gossip-based message dissemination protocol.

We present a performance analysis for image registration with gradient descent methods. We consider a typical multiscale registration setting where the global 2-D translation between a pair of images is estimated by smoothing the images and minimizing the distance between them with gradient descent. Our study particularly concentrates on the effect of noise and low-pass filtering on the alignment accuracy. We adopt an analytic representation for images and analyze the well-behavedness of the image distance function by estimating the neighborhood of translations for which it is free of undesired local minima. This corresponds to the neighborhood of translation vectors that are correctly computable with a simple gradient descent minimization. We show that the area of this neighborhood increases at least quadratically with the smoothing filter size, which justifies the use of a smoothing step in image registration with local optimizers such as gradient descent. We then examine the effect of noise on the alignment accuracy and derive an upper bound for the alignment error in terms of the noise properties and filter size. Our main finding is that the error increases at a rate that is at least linear with respect to the filter size. Therefore, smoothing improves the well-behavedness of the distance function; however, this comes at the cost of amplifying the alignment error in noisy settings. Our results provide a mathematical insight about why hierarchical techniques are effective in image registration, suggesting that the multiscale coarse-to-fine alignment strategy of these techniques is very suitable from the perspective of the trade-off between the well-behavedness of the objective function and the registration accuracy. To the best of our knowledge, this is the first such study for descent-based image registration.

Observational data usually comes with a multimodal nature, which means that it can be naturally represented by a multi-layer graph whose layers share the same set of vertices (objects) with different edges (pairwise relationships). In this paper, we address the problem of combining different layers of the multi-layer graph for an improved clustering of the vertices compared to using layers independently. We propose two novel methods, which are based on a joint matrix factorization and a graph regularization framework respectively, to efficiently combine the spectrum of the multiple graph layers, namely the eigenvectors of the graph Laplacian matrices. In each case, the resulting combination, which we call a “joint spectrum” of multiple layers, is used for clustering the vertices. We evaluate our approaches by experiments with several real world social network datasets. Results demonstrate the superior or competitive performance of the proposed methods compared to state-of-the-art techniques and common baseline methods, such as co-regularization and summation of information from individual graphs.

We present a system for collaborative video streaming in wired overlay networks. We propose a scheme that builds on both rateless codes and network coding in order to improve the system throughput and the video quality at clients. Our hybrid coding algorithm permits to efficiently exploit the available source and path diversity without the need for expensive routing nor scheduling algorithms. We consider specifically an architecture where multiple streaming servers simultaneously deliver video information to a set of clients. The servers apply Raptor coding on the video packets for error resiliency, and the overlay nodes selectively combine the Raptor coded video packets in order to increase the packet diversity in the system. We analyze the performance of selective network coding and describe its application to practical video streaming systems. We further compute an effective source and channel rate allocation in our collaborative streaming system. We estimate the expected symbol diversity at clients with respect to the coding choices. Then we cast a minmax quality optimization problem that is solved by a low-cost bisection based method. The experimental evaluation demonstrates that our system typically outperforms Raptor video streaming systems that do not use network coding as well as systems that perform decoding and encoding in the network nodes. Finally, our solution has a low complexity and only requires small buffers in the network coding nodes, which are certainly two important advantages toward deployment in practical streaming systems.

In this paper, we assume that the network resources are managed by several brokers, which are endowed with resources by a (remote) central resource manager according to several predetermined policies. Our focus is on autonomous multimedia users. We propose a novel resource management scheme, where resource brokers choose well-suited axiomatic bargaining solutions to divide their allocated resources among the users associated with them. These resource division solutions enable resource brokers to provide strict minimum video quality guarantees according to the (varying) number of multimedia users associated with them. Finally, we show that the proposed solution enables us to model the problem of selecting resource brokers by multimedia users as an unweighted congestion game, thereby ensuring convergence to a stationary distribution of users across resource brokers. We investigate the number of required users' switches to reach the stationary distribution, and quantify the fairness of the stationary distribution by introducing a novel quality fairness comparison metric for the users.

In this letter, we consider peer-to-peer (P2P) networks, where multiple peers are interested in sharing their content. In the considered P2P system, autonomous and self-interested peers use a Markov Decision Process (MDP) framework to determine their upload bandwidth allocations, which maximize their individual utilities. This framework enables the peers to make foresighted decisions on their bandwidth allocations, by considering the future impact of their decisions. In this letter, we focus on the impact of the peers’ bounded rationality on their resource reciprocation strategies and ultimately, on their achievable utilities. Specifically, we consider peers who have only a limited ability to model the other peers' strategies for resource reciprocation, and study how this impacts their own decisions.

Conventionally, linear block codes designed for packet erasure correction are targeted to recover all the lost source packets per block, when the fraction of lost data is smaller than the redundancy overhead. However, these codes fail to recover any lost packets, if the number of erasures just exceeds the limit for full recovery capability, while it can still be beneficial to recover part of the symbols. In addition, common linear block codes are not well suited for unequal error protection, since different block codes with different rates must be allocated for each priority class separately. These two problems motivate the design of more flexible forward error correction (FEC) codes for media streaming applications. We first review the performance of short and long linear block codes. Long block codes generally offer better error correction capabilities, but at the price of higher complexity and larger coding delay. Short block codes can be more appropriate in media streaming applications that require smooth performance degradation when the channel loss rate increases. We study a new class of linear block codes using sparse generator matrices that permit to optimize the performance of short block codes for partial recovery of the lost packets. In addition, the proposed codes are extended to the design of unequal erasure protection solutions. Simulations of practical video streaming scenarios demonstrate that the flexible sparse codes offer a promising solution with interesting error correction capabilities and small variance in the residual loss rate. They typically represent an effective trade-off between short block codes with limited flexibility, and long block codes with delay penalties.

The paper addresses the distributed path computation and rate allocation problems for video delivery over multipath networks. The streaming rate on each path is determined such that the end-to-end media distortion is minimized, when a media client aggregates packets received via multiple network channels to the streaming server. In common practical scenarios, it is, however, difficult for the server to have the full knowledge about the network status. Therefore, we propose here a distributed path selection and rate allocation algorithm, where the network nodes participate to the optimized path selection and rate allocation based on their local view of the network. This eliminates the need for end-to-end network monitoring, and permits the deployment of large scale rate allocation solutions. We design a distributed algorithm for optimized rate allocation, where the media client iteratively determines the best set of streaming paths, based on information gathered by network nodes. Each intermediate node then forwards incoming media flows on the outgoing paths, in a distributed manner. The proposed algorithm is shown to quickly converge to the rate allocation that provides a maximal quality to the video client. We also propose a distributed greedy algorithm that achieves close-to-optimal end-to-end distortion performance in a single pass. Both algorithms are shown to outperform simple heuristic-based rate allocation approaches for numerous random network topologies. They offer an interesting solution for media-specific rate allocation over large scale multipath networks.

This paper addresses the problem of efficient representation of scenes captured by distributed omnidirectional vision sensors. We propose a novel geometric model to describe the correlation between different views of a 3-D scene. We first approximate the camera images by sparse expansions over a dictionary of geometric atoms. Since the most important visual features are likely to be equivalently dominant in images from multiple cameras, we model the correlation between corresponding features in different views by local geometric transforms. For the particular case of omnidirectional images, we define the multiview transforms between corresponding features based on shape and epipolar geometry constraints. We apply this geometric framework in the design of a distributed coding scheme with side information, which builds an efficient representation of the scene without communication between cameras. The Wyner–Ziv encoder partitions the dictionary into cosets of dissimilar atoms with respect to shape and position in the image. The joint decoder then determines pairwise correspondences between atoms in the reference image and atoms in the cosets of the Wyner–Ziv image in order to identify the most likely atoms to decode under epipolar geometry constraints. Experiments demonstrate that the proposed method leads to reliable estimation of the geometric transforms between views. In particular, the distributed coding scheme offers similar rate-distortion performance as joint encoding at low bit rate and outperforms methods based on independent decoding of the different images.

Today's packet networks including the Internet offer an intrinsic diversity for media distribution in terms of available network paths and servers or information sources. Novel communication infrastructures such as ad hoc or wireless mesh networks use network diversity to extend their reach at low cost. Diversity can bring interesting benefits in supporting resource greedy applications such as media streaming services, by aggregation of bandwidth and computing resources. Typically, overlay network architectures compensate for lack of quality-of-service guarantees in the network by introducing redundancy in the media delivery system through network diversity. They can support efficient multimedia services when routing, coding, and scheduling algorithms are able to adapt to both the media information and the dynamic network status. This paper presents an overview of the distributed streaming solutions that profit from network diversity in order to improve the quality of multimedia applications. We discuss the coding techniques used for adaptive and flexible media streaming with network diversity. We describe the problem of media streaming with path diversity and focus on routing, path computation, and packet scheduling problems in multipath networks. Then, the advantages of server or source peer diversity in collaborative streaming solutions are discussed. Lastly, we present an overview of wireless mesh networks and focus on the typical constraints imposed by these novel communication models on media streaming with network diversity.

The most expensive part of all electronic structure calculations based on density functional theory lies in the computation of an invariant subspace associated with some of the smallest eigenvalues of a discretized Hamiltonian operator. The dimension of this subspace typically depends on the total number of valence electrons in the system, and can easily reach hundreds or even thousands when large systems with many atoms are considered. At the same time, the discretization of Hamiltonians associated with large systems yields very large matrices, whether with planewave or real-space discretizations. The combination of these two factors results in one of the most significant bottlenecks in computational materials science. In this paper we show how to efficiently compute a large invariant subspace associated with the smallest eigenvalues of a symmetric/Hermitian matrix using polynomially filtered Lanczos iterations. The proposed method does not try to extract individual eigenvalues and eigenvectors. Instead, it constructs an orthogonal basis of the invariant subspace by combining two main ingredients. The first is a filtering technique to dampen the undesirable contribution of the largest eigenvalues at each matrix-vector product in the Lanczos algorithm. This technique employs a well-selected low pass filter polynomial, obtained via a conjugate residual-type algorithm in polynomial space. The second ingredient is the Lanczos algorithm with partial reorthogonalization. Experiments are reported to illustrate the efficiency of the proposed scheme compared to state-of-the-art implicitly restarted techniques.

This paper addresses the problem of streaming packetized media over a lossy packet network, with sender-driven (re)transmissions and receiver acknowledgments. It extends the markovian formulation of the rate- distortion optimized (RaDiO) streaming framework in allowing the transmission schedule of a media data unit to become contingent on the acknowledgements relative to other data units. Media decoding dependencies are generally considered in state-of-the-art rate-distortion optimized scheduling algorithms. However, the set of eligible packet schedules are restricted to independent streaming policies, where the transmission strategy envisioned for a data unit at future transmission opportunities only depends on its own acknowledgment, and not on the acknowledgments potentially received for other data units. This paper questions the validity of this assumption in the design of rate-distortion optimal streaming solutions, and provides a first attempt in the formal derivation of the benefit offered by dependent policies. It considers that the scheduling of a data unit at future transmission opportunities can depend on the acknowledgments received for other data units. One of the main contributions of our paper is to propose a methodology that limits the search space of dependent policies to dependencies that are likely to bring a rate-distortion benefit, in order to solve an optimization problem that is a priori computationally intractable. Extensive simulations validate the proposed approach that focuses on relevant dependencies between streaming policies. We further show that the benefit of dependent streaming policies is actually quite marginal in practical scenarios where the gain in distortion per unit of rate decreases along the media decoding dependency path. It represents the first demonstration that the common assumption of independent streaming policies is valid in many common streaming scenarios. However, experimental results still encourage a careful investigation of dependent policies when the content is characterized by a significant increase of the benefit per transmission unit brought along the data unit dependency path.

In the past few years, the problem of distributed consensus has received a lot of attention, particularly in the framework of ad hoc sensor networks. Most methods proposed in the literature attack this problem by distributed linear iterative algorithms, with asymptotic convergence of the consensus solution. In this letter, we propose the use of extrapolation methods in order to accelerate distributed linear iterations. The extrapolation methods are guaranteed to converge in a finite number of steps, upper bounded by the number of sensors. In particular, we show that the Scalar Epsilon Algorithm (SEA) can accelerate vector sequences produced by distributed linear iterations, with no communication overhead and without knowledge of the full network topology. We provide simulation results that demonstrate the validity and effectiveness of the proposed scheme.

We consider streaming of pre-encoded and packetized media over best-effort networks in presence of acknowledgment feedbacks. We first review the rate-distortion optimization framework in such scenarios. Given an estimation of future transmission resources, and knowing about past transmissions and received acknowledgments, a scheduling algorithm is defined as a mechanism that selects the data to send over the network at any given time, so as to minimize the end-to-end distortion for the given communication resources. Since the computational complexity induced by optimal solution is unacceptable for practical scenarios, we propose to limit the solution search space in using a greedy scheduling strategy. However, our work highlights the rate-distortion sub-optimality of popular greedy schedulers, which are strongly penalized by anticipated retransmissions. We therefore proposes an original scheduling algorithm that avoids premature retransmissions, while preserving the low computational complexity of the greedy paradigm. Such a scheduling strategy allows to adapt to network QoS fluctuations, with close to optimal rate-distortion performance. Our experimental results demonstrate that the proposed patient greedy (PG) scheduler provides a reduction of up to 50% in transmission rate relative to conventional greedy approaches, and that it brings up to 2dB of quality improvement in scheduling classical MPEG-based packet video streams.

Supporting streaming media applications over current packet network infrastructures represents a challenging task in many regards. For one, the lack of quality of service (QoS) guarantees in existing networks such as the Internet means that time-constrained media packets will face dynamic variations in bandwidth, loss rate, and delay as they traverse the network from the sender to the receiver. The variable rate of media traffic represents yet another difficulty when transmission constraints need to be met. Finally, the heterogeneity of client devices and access bandwidth coupled with custom user preferences exacerbate the problem of smooth and quality-optimized media playback even further. In this article we provide an overview of the various techniques for media and streaming strategy adaptation, which can be employed to deal with the difficulties imposed by such dynamic environments. These techniques depend on the characteristics of the media application, in particular on the network streaming infrastructure and the timing constraints imposed on the media packets' delivery. We survey adaptation techniques that act on the encoding of the multimedia information, on the scheduling of the media packets, or that try to combat transmission errors. We also briefly overview some media-friendly networking solutions, which contribute to increased QoS by incorporating some level of intelligence in intermediate network nodes. Finally, we describe a few open challenges in media streaming, emphasizing strategies based on promising cross-layer approaches where adaptation strategies are applied in a coordinated manner, across different layers of the network protocol stack

We address the problem of delay-constrained streaming of multimedia packets over dynamic bandwidth channels. Efficient streaming solutions generally rely on the knowledge of the channel bandwidth, in order to select the media packets to be transmitted, according to their sending time. However, the streaming server usually cannot have a perfect knowledge of the channel bandwidth, and important packets may be lost due to late arrival, if the scheduling is based on an over-estimated bandwidth. Robust media streaming techniques should take into account the mismatch between the values of the actual channel bandwidth and its estimation at the server. We address this rate prediction mismatch by media scheduling with a conservative delay, which provides a safety margin for the packet delivery, even in the presence of unpredicted bandwidth variations. We formulate an optimization problem whose goal is to obtain the optimal value for the conservative delay to be used in the scheduling process, given the network model and the actual playback delay imposed by the client. We eventually propose a simple alternative to the computation of the scheduling delay, which is effective in real-time streaming scenarios. Our streaming method proves to be robust against channel prediction errors, and performs better than other robustness mechanisms based on frame reordering strategies.

This paper presents a progressive coding scheme for 3D objects, based on an overcomplete decomposition of the 3D model on a sphere. Due to increased freedom in the bases construction, redundant expansions have shown interesting approximation properties in the decomposition of signals with multidimensional singularities organized along embedded submanifolds. We propose to map simple 3D models on 2D spheres and then to decompose the signal over a redundant dictionary of oriented and anisotropic atoms that live on the sphere. The signal expansion is computed iteratively with a Matching Pursuit algorithm, which greedily selects the most prominent components of the 3D model. The decomposition therefore inherently represents a progressive stream of atoms, which is advantageously used in the design of scalable representations. An encoder is proposed that compresses the stream of atoms by adaptive coefficient quantization, and entropy coding of atom indexes. Experimental results show that the novel coding strategy outperforms state-of-the-art progressive coders in terms of distortion, mostly at low bit rate. Furthermore, since the dictionary is built on structured atoms, the representation simultaneously offers an increased flexibility. This enables easy stream manipulations, and we finally illustrate this advantage in the design of a view-dependent transmission scheme.

We propose a Rate-Distortion (RD) optimized strategy for frame-dropping and scheduling of multi-user conversa- tional and streaming videos. We consider a scenario where conversational and streaming videos share the forwarding resources at a network node. Two buffers are setup on the node to temporarily store the packets for these two types of video applications. For streaming video, a big buffer is used as the associated delay constraint of the application is moderate and a very small buffer is used for conversational video to ensure that the forwarding delay of every packet is limited. A scheduler is located behind these two buffers that dynamically assigns transmission slots on the outgoing link to the two buffers. Rate-distortion side information is used to perform RD-optimized frame dropping in case of node overload. Sharing the data rate on the outgoing link between the con- versational and the streaming videos is done either based on the fullness of the two associated buffers or on the mean incoming rates of the respective videos. Simulation results showed that our proposed RD-optimized frame dropping and scheduling ap- proach provides significant improvements in performance over the popular priority-based random dropping (PRD) technique.

New breakthroughs in image coding possibly lie in signal decomposition through nonseparable basis functions that can efficiently capture edge characteristics, present in natural images. The work proposed in this paper provides an adaptive way of representing images as a sum of two-dimensional features. It presents a low bit-rate image coding method based on a matching pursuit (MP) expansion, over a dictionary built on anisotropic refinement and rotation of contour-like atoms. This method is shown to provide, at low bit rates, results comparable to the state of the art in image compression, represented here by JPEG2000 and SPIHT, with generally a better visual quality in the MP scheme. The coding artifacts are less annoying than the ringing introduced by wavelets at very low bit rate, due to the smoothing performed by the basis functions used in the MP algorithm. In addition to good compression performances at low bit rates, the new coder has the advantage of producing highly flexible streams. They can easily be decoded at any spatial resolution, different from the original image, and the bitstream can be truncated at any point to match diverse bandwidth requirements. The spatial adaptivity is shown to be more flexible and less complex than transcoding operations generally applied to state of the art codec bitstreams. Due to both its ability for capturing the most important parts of multidimensional signals, and a flexible stream structure, the image coder proposed in this paper represents an interesting solution for low to medium rate image coding in visual communication applications.

This paper presents a highly flexible video coding scheme (MP3D), based on the use of a redundant 3-D spatio-temporal dictionary of functions. Directionality and anisotropic scaling are key ingredients to the spatial components, that form a rich collection of 2-D visual primitives. The temporal component is tuned to capture most of the energy in the temporal signal evolution, along motion trajectories in the video sequences. The MP3D video coding scheme first computes motion trajectories, that are lossless entropy coded and sent as side information to the decoder. It then applies a spatio-temporal decomposition using an adaptive approximation algorithm based on Matching Pursuit (MP). Quantized coefficients and basis function parameters are entropy-coded in a embedded stream that is constructed to respect multiple rate constraints. The geometric properties of the 2-D primitive dictionary allows for flexible spatial resolution adaptation, so that the MP3D stream allows for decoding at multiple rate and spatio-temporal resolutions. The MP3D scheme is shown to provide comparable rate-distortion performances at low and medium bit rates against state-of-the-art schemes, like H.264 and MPEG-4, or the scalable MC EZBC. It also provides an increased flexibility in stream manipulation to adapt to non-octave based spatial resolutions, or to any rate constraints. However, the use of a redundant dictionary is penalizing at high coding rates, which makes the MP3D algorithm interesting for low rate applications, or as a flexible base layer for higher rate video systems.

This paper proposes a rate-distortion optimal a posteriori quantization scheme for Matching Pursuit coefficients. The a posteriori quantization applies to a Matching Pursuit expansion that has been generated off-line, and cannot benefit of any feedback loop to the encoder in order to compensate for the quantization noise. The redundancy of the Matching Pursuit dictionary provides an indicator of the relative importance of coefficients and atom indices, and subsequently on the quantization error. It is used to define a universal upper-bound on the decay of the coefficients, sorted in decreasing order of magnitude. A new quantization scheme is then derived, where this bound is used as an Oracle for the design of an optimal a posteriori quantizer. The latter turns the exponentially distributed coefficient entropy-constrained quantization problem into a simple uniform quantization problem. Using simulations with random dictionaries, we show that the proposed exponentially upper-bounded quantization (EUQ) clearly outperforms classical schemes. Stepping on the ideal Oracle-based approach, a sub-optimal adaptive scheme is then designed that approximates the EUQ but still outperforms competing quantization methods in terms of rate-distortion characteristics. Finally, the proposed quantization method is studied in the context of image coding. It performs similarly to state-of-the-art coding methods (and even better at low rates), while interestingly providing a progressive stream, very easy to transcode and adapt to changing rate constraints.

This paper deals with the optimal allocation of MPEG-2 encoding and media-independent forward error correction (FEC) rates under a total given bandwidth. The optimality is defined in terms of minimum perceptual distortion given a set of video and network parameters. We first derive the set of equations leading to the residual loss process parameters. That is, the packet loss ratio (PLR) and the average burst length after FEC decoding. We then show that the perceptual source distortion decreases exponentially with the increasing MPEG-2 source rate. We also demonstrate that the perceptual distortion due to data loss is directly proportional to the number of lost macroblocks, and therefore decreases with the amount of channel protection. Finally, we derive the global set of equations that lead to the optimal dynamic rate allocation. The optimal distribution is shown to outperform classical FEC scheme, thanks to its adaptivity to the scene complexity, the available bandwidth and to the network performance. Furthermore, our approach holds for any standard video compression algorithms (i.e., MPEG-x, H.26x).

We address the problem of video quality prediction and control for high-resolution video transmitted over lossy packet networks. In packet video, the bitstream flows through several subsystems (coder, network, decoder); each of them can impair the information, either by data loss or by introducing some delay. However, each of these subsystems can be fine- tuned in order to minimize these problems and to optimize the quality of the delivered signal, taking into account the available bitrate. The assessment of the end-user quality is a non-trivial issue. We analyse how the user-perceived quality is related to the average encoding bitrate for variable bit rate MPEG-2 video. We then show why simple distortion metrics may lead to inconsistent interpretations. Furthermore, for a given coder setup, we analyse the effect of packet loss on the user-level quality. We then demonstrate that, when jointly studying the impact of coding bit rate and packet loss, the reachable quality is upperbound and exhibits one optimal coding rate for a given packet loss ratio.

Manifold models provide low-dimensional representations that are useful for analyzing and classifying data in a transformation-invariant way. In this paper we study the problem of jointly building multiple pattern transformation manifolds from a collection of image sets, where each set consists of observations from a class of geometrically transformed signals. We build the manifolds such that each manifold approximates a different signal class. Each manifold is characterized by a representative pattern that consists of a linear combination of analytic atoms selected from a continuous dictionary manifold. We propose an iterative algorithm for jointly building multiple manifolds such that the classification accuracy is promoted in the learning of the representative patterns. We present a DC (Difference-of-Convex) optimization scheme that is applicable to a wide range of transformation and dictionary models, and demonstrate its application to transformation manifolds generated by the rotation, translation and scaling of a reference image. Experimental results suggest that the proposed method yields a high classification accuracy compared to reference methods based on individual manifold building or locally linear manifold approximations.

This paper proposes a novel joint reconstruction algorithm to decode sets of correlated images from distributively compressed images. We consider a scenario where the images captured at different viewpoints are encoded independently using transform-based coding solutions (e.g., SPIHT) with a balanced rate distribution among different cameras. A central decoder jointly processes the compressed images and reconstructs an image pair by exploiting the correlation between images. The central decoder first estimates the underlying correlation model from the independently compressed images and it is eventually used for the joint signal recovery. The joint reconstruction is cast as a constrained convex optimization problem that reconstructs a total-variation (TV) smooth image pair that satisfies with the estimated correlation model. At the same time, we add constraints that force the reconstructed images to be as close as possible to the compressed views. We show by experiments that the proposed joint reconstruction scheme outperforms independent reconstruction in terms of image quality, for a given target bit rate

We consider the problem of distributed average consensus in a sensor network where sensors exchange quantized information with their neighbors. We propose a novel quantization scheme that exploits the increasing correlation between the values exchanged by the sensors throughout the iterations of the consensus algorithm. A low complexity, uniform quantizer is implemented in each sensor, and refined quantization is achieved by progressively reducing the quantization intervals during the convergence of the consensus algorithm. We propose a recurrence relation for computing the quantization parameters that depend on the network topology and the communication rate. We further show that the recurrence relation can lead to a simple exponential model for the size of the quantization step size over the iterations, whose parameters can be computed a priori. Finally, simulation results demonstrate the effectiveness of the progressive quantization scheme that leads to the consensus solution even at low communication rate.

The paper addresses the problem of distributed congestion control in networks where several video stream- ing sessions share joint bottleneck channels. Based on the rate-distortion characteristics of the video streams as well as the requirements of heterogeneous streaming clients, the proposed algorithm determines the utility of rate in- crements in terms of overall quality benefits. We build on the utility-based congestion control framework initially proposed by Kelly (1), (2) and extend it such that it could efficiently handle heterogeneous delays in the network as well as the specific requirements of video streams. We then describe an original implementation of the proposed distributed congestion control algorithm using the common RTP/UDP/IP protocol stack for the efficient streaming of scalable video streams. Each video session independently adapts its streaming rate based on receiver feedbacks, which permits extension to large scale system without central coordination. Finally, we provide extensive simulation results that demonstrate the performance of the proposed solution. It is shown to successfully distribute the network resources among the different sessions such that the overall video quality is maximized. Experiments further demonstrate that the proposed scheme fairly cohabits with TCP, which is certainly an important advantage in today's network architectures.

We consider the problem of failure detection in sensor networks and we propose a new distributed detection algorithm based on Group Testing. We examine the presence of defective sensors by employing tests over locally gathered sensor measurements. Tests are represented with binary messages that sensors exchange over dissemination rounds using a gossip algorithm. We propose a novel probabilistic message design that allows the use of a low complexity decoder. Assuming that the maximum number of defective sensors is much smaller than the total number of sensors, we provide a bound on the number of linearly independent messages required for a successful detection of single or multiple defective sensors. Finally, simulations confirm that the proposed method outperforms algorithms based on random walk message gathering in terms of detection accuracy.

In this work we propose a novel inter-cell interference coordination (ICIC) and resource allocation method. Our aim is to maximize the rate of the worst performing user in all cells. We solve the problem in two phases, inter-cell and intra-cell resource management: first we define an ICIC scheme called interference concentration (ICon) in order to manage resources across cells, then each cell independently performs resource allocation to its users while meeting the constraints imposed by ICon. Finally, we adapt the ICIC method in order to balance the performance achieved in neighboring cells. Differently than Soft Frequency Reuse (SFR), ICon assigns received interference power limits, or the Interference Power Profiles (IPPs) rather than transmit power limits. The IPP determines the interference level the cell tolerates on each band. The intuition behind this change is that the interference to a given cell can be concentrated on a small band, resulting in more efficient use of bandwidth. In the intra-cell resource management phase, each cell allocates power and sub-bands to its users given their location in the cell, maximizing the minimum rate such that the IPP of none of its neighboring cells is violated. In order to balance the performance across all cells we use gradient-like updates to IPPs of cells. Finally we simulate an LTE-like system and compare the performance of our method with reuse 1, static FFR and SFR with proportionally fair scheduling of users in each cell. Static ICon achieves 18% higher 5 percentile rate than reuse 1 which was the best of these methods. Adaptive ICon is found to converge almost immediately, and adds an additional 11% to this gain.

We consider the problem of recovering a set of correlated signals (e.g., images from different viewpoints) from a few linear measurements per signal. We assume that each sensor in a network acquires a compressed signal in the form of linear measurements and sends it to a joint decoder for reconstruction. We propose a novel joint reconstruction algorithm that exploits correlation among underlying signals. Our correlation model considers geometrical transformations between the supports of the different signals. The proposed joint decoder estimates the correlation and reconstructs the signals using a simple thresholding algorithm. We give both theoretical and experimental evidence to show that our method largely outperforms independent decoding in terms of support recovery and reconstruction quality., 11 pages, 3 figures

This paper proposes a joint reconstruction algorithm for compressed correlated images that are given under the form of linear measurements. We consider the particular problem where one image is selected as the reference image and it is used as the side information for decoding the compressed correlated images. These compressed images are given under the form of random measurements that are further quantized and entropy coded. The joint decoder estimates the correlation model based on the geometric transformation of features captured by a structured dictionary. We observe that the high frequency components are not efficiently captured in the estimated image, when the correlation information is used alone for image prediction. Hence, we propose a reconstruction strategy that uses the information in the measurements to recover the missing visual information in the predicted image. The reconstruction is based on an optimization algorithm that enforces the reconstructed image to be consistent with the quantized measurements. We further add additional constraints to ensure that the reconstructed image is close to the image predicted from the correlation estimation. The non-linearities introduced due to quantization are considered on both correlation and reconstruction algorithms in order to improve the performance. Experimental results demonstrate the benefit of the reconstruction algorithm as it brings improved coding performance especially at high rate and outperforms independent coding solutions based on JPEG 2000.

In this paper, we present a pull-based dissemination protocol for efficient distribution of scalable video content in overlay peer-to-peer networks with mesh structures. The proposed protocol employs prioritized network coding, where the network coded packets belong to classes that represent packets of different priorities. For a receiver, the pull procedure begins with the reception of buffer vector messages from the senders, which bring information about the numbers and classes of available packets. The receiver node decides on the rate allocation of the different classes to be requested from each of the senders. The rate allocation is cast as a video quality maximization problem and solved using a hill-climbing algorithm. The simulation results show that the proposed mechanism, which is able to fully adapt to network dynamics, accounts for the unequal packet importances and utilizes the network resources efficiently.

We consider a multi-source delivery system, where Raptor coding at sources and linear network coding in overlay nodes work in concert for efficient data delivery in networks with diversity. Such a combination permits to increase throughput and loss resiliency in multicast scenarios with possibly multiple sources. The network coding operations however change the degree distribution in the set of packets that reach the receivers, so that the low complexity decoding benefits of Raptor codes are unfortunately diminished. We propose in this paper to change the degree distribution at encoder, in such a way that the degree distribution after network coding operations recovers a form that leads to low complexity decoding. We first analyze how the degree distribution of the encoded symbols is altered by network coding operations and losses in a regular network. Then we formulate a geometric optimization problem in order to compute the best degree distribution for encoding at sources, such that the decoding complexity is low and close to Raptor decoders' performance. Simulations show that it is possible to maintain the low complexity decoding performance of Raptor codes even after linear network coding operations, as long as the coding at sources is adapted to the network characteristics.

The construction of low-dimensional models explaining high-dimensional signal observations provides concise and efficient data representations. In this paper, we focus on pattern transformation manifold models generated by in-plane geometric transformations of 2D visual patterns. We propose a method for computing a manifold by building a representative pattern such that its transformation manifold accurately fits a set of given observations. We present a solution for the progressive construction of the representative pattern with the aid of a parametric dictionary, which in turn provides an analytical representation of the data and the manifold. Experimental results show that the patterns learned with the proposed algorithm can efficiently capture the main characteristics of the input data with high approximation accuracy, where the invariance to the geometric transformations of the data is accomplished due to the transformation manifold model.

Mobile phone data provides rich dynamic information on human activities in social network analysis. In this paper, we represent data from two different modalities as a graph and functions defined on the vertex set of the graph. We propose a regularization framework for the joint utilization of these two modalities of data, which enables us to model evolution of social network information and efficiently classify relationships among mobile phone users. Simulations based on real world data demonstrate the potential application of our model in dynamic scenarios, and present competitive results to baseline methods for combining multimodal data in the learning and clustering communities.

This paper proposes a methodology to estimate the correlation model between a pair of images that are given under the form of linear measurements. We consider an image pair whose common objects are relatively displaced due to the positioning of vision sensors. In such scenarios the correlation model that relates the displacement between the objects is effectively represented by a disparity image. We consider a framework where each image is directly acquired and compressed by projecting onto a random basis of lower dimension. Given the linear measurements computed from the images we propose to estimate the underlying correlation model directly in the compressed domain without reconstructing the images that is usually a costly solution. We first show that the correlated images can be efficiently related using a linear operator. Using this linear relationship between the images we derive the relationship between the corresponding measurements in the compressed domain. The underlying correlation model is then built by solving a regularized energy minimization problem. Experimental results show that the proposed scheme estimates an accurate correlation model between the images. Also we show by experiments that the proposed scheme performs competitively with the scheme that estimates the correlation model from the reconstructed images.

We consider the problem of classification of a pattern from multiple compressed observations that are collected in a sensor network. In particular, we exploit the properties of random projections in generic sensor devices and we take some first steps in introducing linear dimensionality reduction techniques in the compressed domain. We design a classification framework that consists in embedding the low dimensional classification space given by classical linear dimensionality reduction techniques in the compressed domain. The measurements of the multiple observations are then projected onto the new classification subspace and are finally aggregated in order to reach a classification decision. Simulation results verify the effectiveness of our scheme and illustrate that compressed measurements combined with information diversity lead to efficient dimensionality reduction in simple sensing architectures.

This paper addresses the reconstruction of high resolution omnidirectional images from multiple low resolution images with inexact registration. When omnidirectional images from low resolution vision sensors can be uniquely mapped on the 2-sphere, such a reconstruction can be described as a transform domain super-resolution problem in the spherical imaging framework. We describe how several spherical images with arbitrary rotations in the SO(3) rotation group contribute to the reconstruction of a high resolution image with help of the Spherical Fourier Transform (SFT). As low resolution images might not be perfectly registered in practice, the impact of inaccurate alignment on the transform coefficients is further analyzed. We then cast the joint registration and super- resolution problem as a total least squares norm minimization problem in the SFT domain. A l1- regularized total least squares problem is also considered. The regularized problem is solved efficiently by interior point methods. Experiments with synthetic and natural images show that the proposed scheme leads to effective reconstruction of high resolution images even when large registration errors exist in the low resolution images. The quality of the reconstructed images also increases rapidly with the number of low resolution images, which demonstrates the benefits of the proposed solution in super-resolution schemes. Finally, we highlight the benefit of the additional regularization constraint that clearly leads to reduced noise and improved reconstruction quality.

Network coding permits to deploy distributed packet delivery algorithms that locally adapt to the network availability in media streaming applications. However, it may also increase delay and computational complexity if it is not implemented efficiently. We address here the effective placement of nodes that implement randomized network coding in overlay networks, so that the goodput is kept high while the delay for decoding stays small in streaming applications. We first estimate the decoding delay at each client, which depends on the innovative rate in the network. This estimation permits to identify the nodes that have to perform coding for a reduced decoding delay. We then propose two iterative algorithms for selecting the nodes that should perform network coding. The first algorithm relies on the knowledge of the full network statistics. The second algorithm uses only local network statistics at each node. Simulation results show that large performance gains can be achieved with the selection of only a few network coding nodes. Moreover, the second algorithm performs very closely to the central estimation strategy, which demonstrates that the network coding nodes can be selected efficiently in a distributed manner. Our scheme shows large gains in terms of achieved throughput, delay and video quality in realistic overlay networks when compared to methods that employ traditional streaming strategies as well as random network nodes selection algorithms., submitted to IEEE Transactions on Multimedia, January 18th 2011

In this paper, we propose to study the problem of localization of a dense set of people with a network of heterogeneous cameras. We propose to recast the problem as a linear inverse problem. The proposed framework is generic to any scene, scalable in the number of cameras and versatile with respect to their geometry, e.g. planar or omnidirectional. It relies on deducing an \emph {occupancy vector}, i.e. the discretized occupancy of people on the ground, from the noisy binary silhouettes observed as foreground pixels in each camera. This inverse problem is regularized by imposing a sparse occupancy vector, i.e. made of few non- zero elements, while a particular dictionary of silhouettes linearly maps these non-empty grid locations to the multiple silhouettes viewed by the cameras network. This constitutes a linearization of the problem, where non- linearities, such as occlusions, are treated as additional noise on the observed silhouettes. Mathematically, we express the final inverse problem either as Basis Pursuit DeNoise or Lasso convex optimization programs. The sparsity measure is reinforced by iteratively re-weighting the $\ell_1$-norm of the occupancy vector for better approximating its $\ell_0$ ``norm'', and a new kind of ``repulsive'' sparsity is used to adapt further the Lasso procedure to the occupancy reconstruction. Practically, an adaptive sampling process is proposed to reduce the computation cost and monitor a large occupancy area. Qualitative and quantitative results are presented on a basketball game. The proposed algorithm successfully detects people occluding each other given severely degraded extracted features, while outperforming state-of-the-art people localization techniques.

The distributed representation of correlated images is an important challenge in applications such as multi-view imaging in camera networks or low complexity video coding. This paper addresses the problem of distributed coding of images whose correlation is driven by the motion of objects or the positioning of the vision sensors. It concentrates on the problem where images are encoded with compressed linear measurements, which are used for estimation of the correlation between images at decoder. We propose a geometry-based correlation model in order to describe the correlation between pairs of images. We assume that the constitutive components of natural images can be captured by visual features that undergo local transformations (e.g., translation) in different images. These prominent visual features are estimated with a sparse approximation of a reference image by a dictionary of geometric basis functions. The corresponding features in the other images are then identified from the compressed measurements. The correlation model is given by the relative geometric transformations between corresponding features. We thus formulate a regularized optimization problem for the estimation of correspondences where the local transformations between images form a consistent motion or disparity map. Then, we propose an efficient joint reconstruction algorithm that decodes the compressed images such that they stay consistent with the quantized measurements and the correlation model. Experimental results show that the proposed algorithm effectively estimates the correlation between images in video sequences or multi-view data. In addition, the proposed reconstruction strategy provides effective decoding performance that compares advantageously to distributed coding schemes based on disparity or motion learning and to independent coding solution based on JPEG-2000.

Transformation-invariant analysis of signals often requires the computation of the distance from a test pattern to a transformation manifold. In particular, the estimation of the distances between a transformed query signal and several transformation manifolds representing different classes provides essential information for the classification of the signal. In many applications, the computation of the exact distance to the manifold is costly, whereas an efficient practical solution is the approximation of the manifold distance with the aid of a manifold grid. In this paper, we consider a setting with transformation manifolds of known parameterization. We first present an algorithm for the selection of samples from a single manifold that permits to minimize the average error in the manifold distance estimation. Then we propose a method for the joint discretization of multiple manifolds that represent different signal classes, where we optimize the transformation-invariant classification accuracy yielded by the discrete manifold representation. Experimental results show that sampling each manifold individually by minimizing the manifold distance estimation error outperforms baseline sampling solutions with respect to registration and classification accuracy. Performing an additional joint optimization on all samples improves the classification performance further. Moreover, given a fixed total number of samples to be selected from all manifolds, an asymmetric distribution of samples to different manifolds depending on their geometric structures may also increase the classification accuracy in comparison with the equal distribution of samples.

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions ( DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.

We present a new descriptor and feature matching solution for omnidirectional images. The descriptor builds on the log-polar planar descriptors, but adapts to the specific geometry and non-uniform sampling density of spherical images. We further propose a rotation-invariant matching method for the proposed descriptor that is particularly interesting for mobile devices. It permits to reduce the computational complexity in the detection phase by eliminating the orientation assignment and to shift it to the feature matching step. We then use a criteria based on the Kullback- Leibler divergence in order to improve the feature matching performance. Experimental results with spherical images show that the new descriptors offer promising performance and improve on SIFT descriptors computed on the sphere or on tangent planes.

Transformation manifolds are quite attractive for image analysis applications that require transformation invariance properties. The geometric structure of a transformation manifold has a profound influence on the design of processing algorithm, and the curvature is a major parameter in the characterization of the manifold geometry. We propose here a procedure for the computation of an upper bound for the maximum principal curvature of a pattern transformation manifold. We provide an analytical formulation of the curvature bound and show that the numerical computation of this bound is mostly dependent on the rotation parameters. Experimental results indicate that the curvature bound of the manifold has considerable dependence on the spatial complexity and smoothness of the generating pattern. Moreover, experiments with discretization of manifolds suggest that the curvature of the manifold is likely to affect the accuracy of compact representation and sampling algorithms.

This paper addresses the problem of clustering heterogeneous clients in IPTV services over lossy networks. The delivery of the same stream to clients with different capabilities or access networks is surely suboptimal in terms of average quality for the population of receivers. Instead, we propose that the streaming servers deliver distinct multicast streams to different subsets of clients. We formulate an optimization problem where the receivers are clustered depending on the quality of their connection so that the average video quality in the IPTV system is maximized. Then we propose a novel algorithm for determining optimally the clusters, as well as the source and channel rate allocation in each of the clusters. Simulation results show that the proposed algorithm is able to maximize the average quality in the system when each of the servers transmits information to a distinct cluster. In particular, we show that the proposed solution outperforms baseline schemes that serve all clients with the same multicast stream, as it is commonly the case in practical systems.

We propose a new method for imaging sound speed in breast tissue from measurements obtained by ultrasound tomography (UST) scan- ners. Given the measurements, our algorithm finds a sparse image representation in an overcomplete dictionary that is adapted to the properties of UST images. This dictionary is learned from high reso- lution MRI breast scans using an unsupervised maximum likelihood dictionary learning method. The proposed dictionary-based regular- ization method significantly improves the quality of reconstructed breast UST images. It outperforms the wavelet-based reconstruction and the least squares minimization with lowpass constraints, on both numerical and in vivo data. Our results demonstrate that the use of the learned dictionary improves the image accuracy for up to 4 dB with the exact measurement matrix and for 3.5 dB with the estimated measurement matrix over the wavelet-based reconstruction under the same conditions.

Novel network architectures such as overlay networks offer significant diversity that can compensate for the lack of strict quality of service in today's communication infrastructures. In order to take advantage of this diversity for delay-sensitive media streaming applications, the network systems can employ efficient mechanisms based on source, channel and even network coding. In particular, fountain codes offer interesting benefits for streaming with server diversity. When they are used independently at each server, they permit to avoid explicit coordination between the senders that only have to provide the receivers with enough innovative packets. In addition, network coding allows for improved throughput and error robustness in multipath transmission where the network nodes participate to increase the symbol diversity in the system. We review in this chapter the most popular rateless codes that enable the deployment of low-cost decentralized communication protocols in self-organized dynamic networks. We then describe their application in distributed multimedia streaming solutions. We further discuss the most popular network coding algorithms in practical media streaming schemes. Finally, we show that hybrid systems based on both rateless coding and network coding can deliver high quality media streams with low computational complexity, as they permit to benefit from both server and path diversity in overlay architectures.