Search

The MPEG-4 video standard extends the traditional frame-based processing with the option to compose several video objects (VO) superimposed on a background sprite image. In our previous work, we presented a distributed, multiprocessor based, scalable implementation of an MPEG-4 arbitrary-shaped decoder, which forms together with the background sprite decoder an essential part for further scene rendering. For control of the multiprocessor architecture, we have constructed a Quality-of-Service (QoS) management that monitors the availability of required data and distributes the processing of individual tasks with guaranteed or best-effort services of the platform. However, the proposed architecture with the combined guaranteed and best-effort services poses problems for real-time scene rendering. In this paper, we present a technique for proper run-time rendering of the final scene after decoding one VO Layer. The individual video-object monitors check the data availability and select the highest quality for the final scene rendering. The algorithm operates hierarchically both at the scene level and at the task level of the video object processing. Whereas the earlier work on scalable implementation concentrated only on guaranteed services, we now introduce a new element in the system architecture for the real-time control and fall back mechanism of the best-effort services. This element is based on first, controlling data availability at task level, and second, introducing the propagation service to QoS management. We present our simulation results in the comparison with the standard "frame-skipping" technique that is the only currently available solution to this type of rendering a scalable processing.

MPEG-4 is the first standard that combines synthetic objects, like 2D/3D graphics objects, with natural rectangular and non-rectangular video objects. The independent access to individual synthetic video objects for further manipulation creates a large space for future applications. This paper addresses the optimization of such complex multimedia algorithms for implementation on multiprocessor platforms. It is shown that when choosing the correct granularity of processing for enhanced parallelism and splitting time-critical tasks, a substantial improvement in processing efficiency can be obtained. In our work, we focus on non-rectangular (also called arbitrary-shaped) video objects decoder. In previous work, we motivated the use of a multiprocessor System-on-Chip(SoC) setup that satisfies the requirements on the overall computation capacity. We propose the optimization of the MPEG-4 algorithm to increase the decoding throughput and a more efficient usage of the multiprocessor architecture. First, we present a modification of the Repetitive Padding to increase the pipelining at block level. We identified the part of the padding algorithm that can be executed in parallel with the DCT-coefficient decoding and modified the original algorithm into two communicating tasks. Second, we introduce a synchronization mechanism that allows the processing for the Extended Padding and postprocessing (Deblocking & Deringing) filters at block level. The first optimization results in about 58% decrease of the original Repetitive-Padding task computational requirements. By introducing the previously proposed data-level parallelism and exploiting the inherent parallelism between the separated color components (Y, Cr, Cb), the computational savings are about 72% on the average. Moreover, the proposed optimizations marginalize the processing latency from frame size to slice order-of-magnitude.

Multiprocessor systems are rapidly entering various high-performance computing segments, like multimedia processing. Instead of an increase in processor clock frequency, the new trend is enabling multiple cores in performing processing, e.g. dual or quadrapule CPUs in one subsystem. In this contribution, we address the problem of modeling the resource requirements of multimedia applications for a distributed computation on a multiprocessor system. This paper shows that the estimation of resource requirements based on input data enables the dynamic activation of tasks and run-time redistribution of application tasks. We also formally specify the optimal selection of the co-executed application with aim to provide the most optimal end-results of such streaming applications within one networks-on-chip (NoC) system. We present a new concept for system optimization which involves the major system parameters and resource usage. Experimental results are based on mapping an arbitrary-shaped MPEG-4 video decoder onto a multiprocessor NoC.

With the growing complexity of new multimedia applications and the parallel execution enabled by multiprocessor architectures, Quality-of-Service (QoS) management for modern Network-on-Chip (NoC) systems becomes relevant. This paper presents an approach for efficient usage of platform resources and simultaneously controlling overall system performance, when multiple applications are active. Therefore, on top of the con- ventional QoS, we introduce an algorithm for setting quality levels of parallel executed applications on a multiprocessor Network-on-Chip. The new concept is a hierarchical system, where an application manager negotiates with the resource management to jointly optimize the quality of individual and overall application settings. The proposed QoS manager concept was successfully evaluated by an experimental set-up, employing two independent video-object decoders, a background sprite decoder from the MPEG-4 standard and an abstract audio object. This simultaneous execution of four independent applications was mapped on a simulator based on AEthereal network and eight ARM7 processors executed with a clock-cycle true simulator. The proposed distributed QoS management decreases the complexity of the overall management and improves the transparency and responsibilities in the decision taking.

Quality-of-Service (QoS) management is becoming an important part of Consumer Electronics systems. The complexity of recent multimedia applications and their requirements on the system throughput ask for a parallel programming model which enables parallel task execution. In this paper, we describe a hierarchical QoS model for managing multimedia applications running on a multiprocessor Systems-on-Chip (SoC). We focus on an MPEG-4 shape-texture decoder that is fully object based, using arbitrary object shapes. We specify the QoS problem within the given application domain. First, we explore the possible application scalability. The obtained results save up to 26% of resources. Second, and in contrast with many existing systems, we employ a novel prediction technique for the execution time of video-object decoding in a group of frames, where the actual execution is spent only on data that have a guarantee to be presented.

The input data dependency is particularly important for advanced multimedia applications, where video content is partitioned into independent video objects instead of rectangular frames of samples. As objects can change over time, in size, shape and texture contents, the processing requirements are more variable then with conventional video. For this reason we have adopted the MPEG-4 standard as an example of object oriented video processing. In this paper we consider a class of QoS systems that relies on predicting the execution times of the application at run-time taking the data dependency into account. We propose a control mechanism based on a linear prediction model that interacts with a higher level QoS management unit to adopt quality and resource settings for our application. In initial experiments with our new concept, we found that the savings in computation is at least in the order of several tenths of percents.

Development costs and time-to-market of software-intensive media processing systems can be reduced when correct decisions are made at an early design phase. However, the real-time requirements imposed on these systems, such as frame skipping and latency limitations, can only be validated after system implementation, e.g. at the test phase. To avoid system redesign for ensuring real-time performance, a system performance analysis should be done as soon as possible. This paper addresses a scenario simulation approach for performance predictions already at the design phase. This approach is presented through a real case study, for which we developed an advanced MPEG-4 coding application. The benefits of the approach are threefold: a) high accuracy of the predicted performance data; b) an efficient real-time software-hardware implementation of a system, because the generic computational costs become known in advance, and c) improved ease of use because of a high abstraction level of modelling. Experiments showed that the prediction accuracy of the system performance is within 10%, while the prediction accuracy of the time-detailed processor usage (performance) does not exceed 30%. However, the real-time performance requirements are sometimes not met, e.g. when other applications require intensive memory usage, thereby imposing delays on the decoding data retrieval from memory.

Background sprite decoding is an essential part of object-based video coding.The composition and rendering of a final scene involves the placing of individual video objects in a predefined way superimposed on the decoded background image. The MPEG-4 standard includes the decoding algorithm for background image decoding, but this algorithm is not suitable for implementation on a memory-constrained platform. In this paper we present a modification of the decoding algorithm that decodes MPEG-4 sequences while fulfilling the requirements of a memory-constrained multiprocessor system with only 17% extra overhead of computation. Our algorithm reduces the memory cost of such decoding with a factor of four. Additionally, our algorithm offers the possibility of high level data parallelism and consequently contributes to an increase of throughput rate.

This paper addresses an efficient implementation of new emerging video algorithms like the coding of arbitrarily shaped video objects in the new MPEG-4 standard. This type of advanced multimedia applications pose challenging requirements on embedded systems design with respect to decomposition and scalability, in order to meet real-time constraints. We study the design of networks-on-chip (NoC), which intrinsically satisfies these requirements [5]. A job scheduler needs to know the worst-case execution time (WCET) of a starting job to ensure that the job can meet its timing constraints. For the purpose of timing analysis, such as computing the WCET, a timing model has been applied which has a linear dependence on a set of inputdependent data parameters. We derive a linear timing model for MPEG-4 video object decoding from a running executable specification. Our timing model is computed and verified with an instruction-set simulator of a RISC processor element containing a flat local memory model. The derived model is accurate within 6% for the average execution time.