Designing autonomous layered video coders

Layered compression paradigms such as scalable, multiple description, and multi-view video coding, require coupled encoding decisions among layers to achieve optimal distortion performance under buffer constraints. Moreover, due to the dynamic and time-varying source characteristics, and temporal coupling of encoding decisions through the buffer constraints, it is not only necessary to consider the immediate rate-distortion impact of encoding decisions, but also their long-term rate-distortion impact. In other words, optimal encoding decisions must consider the coupling between layers and the coupling across time. In many scenarios, however, it may be impractical to make joint coding decisions for all of the layers. For instance, a two layer bitstream may be coded using different encoders for the base layer and enhancement layer, each with its own autonomous control plane; or, if the same encoder is used for multiple layers, then a joint decision process, which considers the aforementioned dependencies, may be too complex. In this paper, we propose a framework for autonomous decision making in layered video coders, which decouples the decision making processes at the various layers using a novel layered Markov decision process. We illustrate how this framework can be applied to decompose the decision processes for several typical layered video coders with different dependency structures and we observe that the performance of the proposed decomposition highly depends on the ability of the layers to model each other.