Search

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.

In large-scale automatic traffic sign surveying systems, the primary computational effort is concentrated at the traffic sign detection stage. This paper focuses on reducing the computational load of particularly the sliding window object detection algorithm which is employed for traffic sign detection. Sliding-window object detectors often use a linear SVM to classify the features in a window. In this case, the classification can be seen as a convolution of the feature maps with the SVM kernel. It is well known that convolution can be efficiently implemented in the frequency domain, for kernels larger than a certain size. We show that by careful reordering of sliding-window operations, most of the frequency-domain transformations can be eliminated, leading to a substantial increase in efficiency. Additionally, we suggest to use the overlap-add method to keep the memory use within reasonable bounds. This allows us to keep all the transformed kernels in memory, thereby eliminating even more domain transformations, and allows all scales in a multiscale pyramid to be processed using the same set of transformed kernels. For a typical sliding-window implementation, we have found that the detector execution performance improves with a factor of 5.3. As a bonus, many of the detector improvements from literature, e.g. chi-squared kernel approximations, sub-class splitting algorithms etc., can be more easily applied at a lower performance penalty because of an improved scalability.

A single stage blind source extraction (BSE) approach to extract one desired source signal from an instantaneous mixture of several signals is studied in this paper. In contrast to methods based on blind source separation (BSS) followed by a classifier we extract directly the desired source signal. In recent papers it has been shown that prior information about the desired source's mixing column or autocorrelation function can be incorporated in a single stage BSE algorithm. The focus of this paper is to give insight in the BSE problem for scenarios where joint diagonalization of correlation matrices is applicable, i.e., to exploit the full potential of the SOS of observed signals. In this work we unify and expand these previously presented BSE methods, which lead to the unified instantaneous blind source extraction (UIBSE) algorithm. We show that UIBSE is even more flexible than previously presented since both types of prior information may be combined to obtain better results in selecting the desired source. Above that, the same algorithm may be used for different objective functions. Finally, a performance comparison with BSS algorithms that are followed by a classifier shows that UIBSE is more robust to noisy observations.

We present an approach for efficient rendering and transmitting views to a high-resolution autostereoscopic display for medical purposes. Displaying biomedical images on an autostereoscopic display poses different requirements than in a consumer case. For medical usage, it is essential that the perceived image represents the actual clinical data and offers sufficiently high quality for diagnosis or understanding. Autostereoscopic display of multiple views introduces two hurdles: transmission of multi-view data through a bandwidth-limited channel and the computation time of the volume rendering algorithm. We address both issues by generating and transmitting limited set of views enhanced with a depth signal per view. We propose an efficient view interpolation and rendering algorithm at the receiver side based on texture+depth data representation, which can operate with a limited amount of views. We study the main artifacts that occur during rendering-occlusions, and we quantify them first for a synthetic model and then for real-world biomedical data. The experimental results allow us to quantify the peak signal-to-noise ratio for rendered texture and depth as well as the amount of disoccluded pixels as a function of the angle between surrounding cameras.

Current development of 3D technologies brings 3DTV within reach for the customers. We discuss in this article the recent advancements in free-viewpoint interpolation for 3D video. This technology is still a research topic and many efforts are dedicated to creation, evaluation and improvement of new algorithms. First, we describe a recent DIBR algorithm that allows free-viewpoint interpolation. We perform detailed analysis of its performance and establish the sources of errors that occur in the resulting 3D videos. Our graph presents each DIBR step’s contribution to the overall error. We show that the challenge consists of finding the correct pixel values in the areas of resulting image where warping did not provide any information. The quality of the resulting image is mostly determined by the warping procedure and cannot be considerably changed by post-processing. To accommodate the generation of stereoscopic videos, we discuss the possibilities of creating a free-viewpoint stereoscopic video.

Multi-view 3D video may succeed stereo 3DTV in multimedia and TV applications. The MPEG committee has installed a special task force to establish a standard for multiview 3D video coding. Enabling free-viewpoint selection by the consumer is one of the major purposes of multi-view 3D encoded data. One focal point of our research work concerns an efficient implementation of the rendering part of stereoscopic free-viewpoint visualization, which is a computationally expensive task and significantly determines the final image quality. The free-viewpoint DIBR algorithm that is described in this article is implemented with an off-theshelf GPU that can be integrated in advanced 3DTV systems. We present the principal steps of a representative freeviewpoint DIBR, based on image warping and image artifact reduction techniques, and show the key differences between the reference software and our GPU implementation. One of those differences is the joint execution of signal processing blocks to share memory usage. Using a combination of the highly parallel programming architecture CUDA and a graphics API, we have achieved real-time performance executing 1080p HD multi-view video with a rendering quality that is comparable to the software implementation.

The term smart environment refers to physical spaces equipped with sensors feeding into adaptive algorithms that enable the environment to become sensitive and responsive to the presence and needs of its occupants. People with special needs, such as the elderly or disabled people, stand to benefit most from such environments as they offer sophisticated assistive functionalities supporting independent living and improved safety. In a smart environment, the key issue is to sense the location and identity of its users. In this paper, we intend to tackle the problems of detecting and tracking humans in a realistic home environment by exploiting the complementary nature of (synchronized) color and depth images produced by a low-cost consumer-level RGB-D camera. Our system selectively feeds the complementary data emanating from the two vision sensors to different algorithmic modules which together implement three sequential components: (1) object labeling based on depth data clustering, (2) human re-entry identification based on comparing visual signatures extracted from the color (RGB) information, and (3) human tracking based on the fusion of both depth and RGB data. Experimental results show that this division of labor improves the system?s efficiency and classification performance.

This paper presents an algorithm for localization and classification of subtitles in TV videos. We extend an existing static-region detector with object-based adaptive temporal filtering, bounding box computation around blobs of refined static regions, bounding box categorization based on geometry and filling degree of static regions, and subtitle classification using text-stroke alignment features. On a test set of more than 5000 video frames, a Precision rate of 96% is achieved at 98% Recall rate. The system detects subtitles without frame delays, and uses techniques suitable for implementation in a TV platform. We also experimentally show that the picture quality of Motion-Compensated Picture Rate Conversion in televisions can benefit from our system.

Watching sports, such as tennis and soccer, remains popular for a broad class of consumers. However, audiences might not be able to enjoy their favorite games from a television or PC when they are traveling, so mobile devices are increasingly used for watching sports video. Unfortunately, watching sports on a mobile device is not as simple as watching sports on TV. Bandwidth limitations on wireless networks prevent high bit-rate video transmission. In addition, small displays lose visual details of the sports event. Bandwidth limitations occur primarily when multiple users want to stream video on the same wireless link. These bottlenecks are likely to occur because of the popularity of an event. This article describes a camera modeling based, mixed-reality system concept. The idea is to build a 3D model for sports video, where all parameters of this model can be obtained by analyzing the broadcasting video. Instead of sending original images to the mobile device, the system only sends parameters of the 3D model and the information about players and balls, which can significantly save transmission bandwidth. Additionally, because we have full-quality information about camera modeling and also players and balls, the mobile client is able to recover the important information without loss of the visual details. In addition, we can generate virtual scenes for less important areas, such as the playing field and the commercial billboard, without changing the major story of the sports game.

In this paper, we consider the problem of finding and localizing social human groups in videos, which can form a basis for further analysis and monitoring of groups in general. Our approach is motivated by the collective behavior of individuals which has a fundament in sociological studies. We design a detection-based multi-target tracking framework which is capable of handling short-term occlusions and producing stable trajectories. Human groups are discovered by clustering trajectories of individuals in an agglomerative fashion. A novel similarity function related to distances between group members, robustly measures the similarity of noisy trajectories. We have evaluated our approach on several test sequences and achieved acceptable miss rates (19.4%, 29.7% and 46.7%) at reasonable false positive detections per frame (0.129, 0.813 and 0.371). The relatively high miss rates are caused by a strict evaluation procedure, whereas the visual results are quite acceptable.

Brain-computer interface (BCI) systems based on the steady-state visual evoked potential (SSVEP) provide higher information throughput and require shorter training than BCI systems using other brain signals. To elicit an SSVEP, a repetitive visual stimulus (RVS) has to be presented to the user. The RVS can be rendered on a computer screen by alternating graphical patterns, or with external light sources able to emit modulated light. The properties of an RVS (e.g., frequency, color) depend on the rendering device and influence the SSVEP characteristics. This affects the BCI information throughput and the levels of user safety and comfort. Literature on SSVEP-based BCIs does not generally provide reasons for the selection of the used rendering devices or RVS properties. In this paper, we review the literature on SSVEP-based BCIs and comprehensively report on the different RVS choices in terms of rendering devices, properties, and their potential influence on BCI performance, user safety and comfort.

A fundamental challenge limiting information quality obtained from smart sensing garments is the influence of textile movement relative to limbs. We present and validate a comprehensive modeling and simulation framework to predict recognition performance in casual loose-fitting garments. A statistical posture and wrinkle-modeling approach is introduced to simulate sensor orientation errors pertained to local garment wrinkles. A metric was derived to assess fitting, the body-garment mobility. We validated our approach by analyzing simulations of shoulder and elbow rehabilitation postures with respect to experimental data using actual casual garments. Results confirmed congruent performance trends with estimation errors below 4% for all study participants. Our approach allows to estimate the impact of fitting before implementing a garment and performing evaluation studies with it. These simulations revealed critical design parameters for garment prototyping, related to performed body posture, utilized sensing modalities, and garment fitting. We concluded that our modeling approach can substantially expedite design and development of smart garments through early-stage performance analysis.

This paper aims at generating an automated way to evaluate the team-behavior of trainees in a delivery simulation course using video-processing techniques with emphasis on multiple people tracking. The paper is composed of two interacting, but clearly separated stages: moving people segmentation and multiple people tracking. At people segmentation stage, the combination of the Gaussian Mixture Model (GMM) and the Dynamic Markov Random Fields (DMRF) technique helps to extract the foreground pixels. For a better extraction of the human silhouettes, the energy function of DMRF is extended with texture information. At multiple people tracking stage, we concentrate on solving human-occlusion problem caused by interacting persons based on silhouette data and a non-linear regression model. Our model effectively transfers the person location problem during the occlusion into the finding of the local maximum points on a smooth curve, so that visual persons in the partial or complete occlusion can still be precisely captured. We have compared our algorithm with two other popular tracking algorithms: mean-shift and particle-filter. Experimental results reveal that the correctness of our method is much higher than the mean-shift algorithm and slightly lower than a particle-filter, however, with the major benefit of being a factor of 10---15 faster in computing.

Although the problems of optical character recognition for contemporary printed text have been resolved, for historical printed and handwritten connected cursive text (i.e. western style writing), they have not. This does not mean that scanning historical documents is not useful. This article describes our research on retrieving digitized handwritten documents containing the information that the user is looking for. This task is essential for optimizing the archive's work. We investigated how to process historical documents and their transcriptions, so that a super computer could learn how to read. We applied artificial intelligence techniques to a large amount of image data and created a search engine. Our methods often require that the computer learns in interaction with a human. We have studied the requests of archive users in order to bring our research as close as possible to the current information needs. Our system learns continuously, allowing the constant improvement of search results. User requests stimulated us to delve into an unsolved topic: to search for the most elusive knowledge in text, namely the names of people and places. The solutions are described in this article.

On-axis and far-field series expansions are developed for the sound pressure due to an arbitrary, circular symmetric velocity distribution on a flat radiator in an infinite baffle. These expansions are obtained by expanding the velocity distributions in terms of orthogonal polynomials R(2n) (0)(sigma/a)=P(n)(2(sigma/a)(2)-1) with P(n) the Legendre polynomials. The terms R(2n) (0) give rise to a closed-form expression for the pressure on-axis as well as for the far-field pressure. Furthermore, for a large number of velocity profiles, including those associated with the rigid piston, the simply supported radiator, and the clamped radiators as well as Gaussian radiators, there are closed-form expressions for the required expansion coefficients. In particular, for the rigid, simply supported, and clamped radiators, this results in explicit finite-series expressions for both the on-axis and far-field pressures. In the reverse direction, a method of estimating velocity distributions from (measured) on-axis pressures by matching in terms of expansion coefficients is proposed. Together with the forward far-field computation scheme, this yields a method for far-field loudspeaker assessment from on-axis data (generalized Keele scheme). The forward computation scheme is extended to dome-shaped radiators with arbitrary velocity distributions.

A major challenge in using multi-modal, distributed sensor systems for activity recognition is to maintain a temporal synchronization between individually recorded data streams. A common approach is to use well defined 'synchronization actions' performed by the user to generate, easily identifiable pattern events in all recorded data streams. The events are then used to manually align data streams. This paper proposes an automatic method for this synchronization. We demonstrate that synchronization actions can be automatically identified and used for stream synchronization across widely different sensors such as acceleration, sound, force, and a motion tracking system. We describe fundamental properties and bounds of our event-based synchronization approach. In particular, we show that the event timing relation is transitive for sensor groups with shared members. We analyzed our synchronization approach in three studies. For a large dataset of 5 users and totally 308 data stream minutes we achieved a synchronization error of 0.3 s for more than 80% of the stream.

This paper addresses privacy leakage in biometric secrecy systems. Four settings are investigated. The first one is the standard Ahlswede-Csiszar secret-generation setting in which two terminals observe two correlated sequences. They form a common secret by interchanging a public message. This message should only contain a negligible amount of information about the secret, but here, in addition, we require it to leak as little information as possible about the biometric data. For this first case, the fundamental tradeoff between secret-key and privacy-leakage rates is determined. Also for the second setting, in which the secret is not generated but independently chosen, the fundamental secret-key versus privacy-leakage rate balance is found. Settings three and four focus on zero-leakage systems. Here the public message should only contain a negligible amount of information on both the secret and the biometric sequence. To achieve this, a private key is needed, which can only be observed by the terminals. For both the generated-secret and the chosen-secret model, the regions of achievable secret-key versus private-key rate pairs are determined. For all four settings, the fundamental balance is determined for both unconditional and conditional privacy leakage.

Power series expansions in ka are derived for the pressure at the edge of a radiator, the reaction force on the radiator, and the total radiated power arising from a harmonically excited, resilient, flat, circular radiator of radius a in an infinite baffle. The velocity profiles on the radiator are either Stenzel functions (1-(sigma/a)2)n, with sigma the radial coordinate on the radiator, or linear combinations of Zernike functions Pn(2(sigma/a)2-1), with Pn the Legendre polynomial of degree n. Both sets of functions give rise, via King's integral for the pressure, to integrals for the quantities of interest involving the product of two Bessel functions. These integrals have a power series expansion and allow an expression in terms of Bessel functions of the first kind and Struve functions. Consequently, many of the results in [M. Greenspan, J. Acoust. Soc. Am. 65, 608-621 (1979)] are generalized and treated in a unified manner. A foreseen application is for loudspeakers. The relation between the radiated power in the near-field on one hand and in the far field on the other is highlighted.

Blind identification is a crucial subtask in signal processing problems such as blind signal separation (BSS) and direction-of-arrival (DOA) estimation. This paper presents a procedure for multiple-input multiple-output instantaneous blind identification based on second-order temporal properties of the signals, such as coloredness and nonstationarity. The procedure consists of two stages. First, based on assumptions on the second-order temporal structure (SOTS) of the source and noise signals, and using subspace techniques, the problem is reformulated in a particular way such that each column of the unknown mixing matrix satisfies a system of multivariate homogeneous polynomial equations. Then, this nonlinear system of equations is solved by means of a so-called homotopy method, which provides a general tool for solving (possibly nonexact) systems of nonlinear equations by smoothly deforming the known solutions of a simple start system into the desired solutions of the target system. Our blind identification procedure allows to estimate the mixing matrix for scenarios with more sources than sensors without resorting to sparsity assumptions, something that is often believed to be impossible when using only second-order statistics. In addition, since our algorithm does not require any assumption on the mixing matrix, also mixing matrices that are rank-deficient or even have identical columns can be identified. Finally, we give examples and performance results for speech source signals.

Hands-free devices are often used in a noisy and reverberant environment. Therefore, the received microphone signal does not only contain the desired near-end speech signal but also interferences such as room reverberation that is caused by the near-end source, background noise and a far-end echo signal that results from the acoustic coupling between the loudspeaker and the microphone. These interferences degrade the fidelity and intelligibility of near-end speech. In the last two decades, post filters have been developed that can be used in conjunction with a single microphone acoustic echo canceller to enhance the near-end speech. In previous works, spectral enhancement techniques have been used to suppress residual echo and background noise for single microphone acoustic echo cancellers. However, dereverberation of the near-end speech was not addressed in this context. Recently, practically feasible spectral enhancement techniques to suppress reverberation have emerged. In this paper, we derive a novel spectral variance estimator for the late reverberation of the near-end speech. Residual echo will be present at the output of the acoustic echo canceller when the acoustic echo path cannot be completely modeled by the adaptive filter. A spectral variance estimator for the so-called late residual echo that results from the deficient length of the adaptive filter is derived. Both estimators are based on a statistical reverberation model. The model parameters depend on the reverberation time of the room, which can be obtained using the estimated acoustic echo path. A novel postfilter is developed which suppresses late reverberation of the near-end speech, residual echo and background noise, and maintains a constant residual background noise level. Experimental results demonstrate the beneficial use of the developed system for reducing reverberation, residual echo, and background noise.

This paper addresses the automatic analysis of court-net sports video content. We extract information about the players, the playing-field in a bottom-up way until we reach scene-level semantic concepts. Each part of our framework is general, so that the system is applicable to several kinds of sports. A central point in our framework is a camera calibration module that relates the a-priori information of the geometric layout in the form of a court model to the input image. Exploiting this information, several novel algorithms are proposed, including playing-frame detection, players segmentation and tracking. To address the player-occlusion problem, we model the contour map of the player silhouettes using a nonlinear regression algorithm, which enables to locate the players during the occlusions caused by players in the same team. Additionally, a Bayesian-based classifier helps to recognize predefined key events, where the input is a number of real-world visual features. We illustrate the performance and efficiency of the proposed system by evaluating it for a variety of sports videos containing badminton, tennis and volleyball, and we show that our algorithm can operate with more than 91% feature detection accuracy and 90% event detection.

Background Laparoscopic surgery is normally performed under general anaesthesia, but regional techniques have been found beneficial, usually in the management of patients with major medical problems. Encouraged by such experience, we performed a feasibility study of segmental spinal anaesthesia in healthy patients. Methods Twenty ASA I or II patients undergoing elective laparoscopic cholecystectomy received a segmental (T10 injection) spinal anaesthetic using 1 ml of bupivacaine 5 mg ml −1 mixed with 0.5 ml of sufentanil 5 µg ml −1 . Other drugs were only given (systemically) to manage patient anxiety, pain, nausea, hypotension, or pruritus during or after surgery. The patients were reviewed 3 days postoperatively by telephone. Results The spinal anaesthetic was performed easily in all patients, although one complained of paraesthesiae which responded to slight needle withdrawal. The block was effective for surgery in all 20 patients, six experiencing some discomfort which was readily treated with small doses of fentanyl, but none requiring conversion to general anaesthesia. Two patients required midazolam for anxiety and two ephedrine for hypotension. Recovery was uneventful and without sequelae, only three patients (all for surgical reasons) not being discharged home on the day of operation. Conclusions This preliminary study has shown that segmental spinal anaesthesia can be used successfully and effectively for laparoscopic surgery in healthy patients. However, the use of an anaesthetic technique involving needle insertion into the vertebral canal above the level of termination of the spinal cord requires great caution and should be restricted in application until much larger numbers of patients have been studied.

A multiphasic septal motion and typical septal-to-lateral apical shuffle of the left ventricle can be observed echocardiographically in some patients with left branch bundle block. The relation of both with left ventricular (LV) dyssynchrony according to tissue Doppler and LV reverse remodeling after cardiac resynchronization therapy was investigated. Fifty-three patients (37 men; age 68 ± 8 years) with ischemic (n = 26) or idiopathic (n = 27) cardiomyopathy, baseline QRS duration 171 ± 30 ms, LV ejection fraction 21 ± 7%, and LV end-diastolic volume 257 ± 91 ml were studied. LV dyssynchrony using tissue Doppler was considered present if the SD of the interval between QRS and onset of systolic velocity of 6 basal LV segments was >20 ms. Shuffle was evaluated visually independently by 5 cardiologists and considered present if observed in =1 view. LV reverse remodeling, defined as LV end-systolic volume decrease =10%, was observed in 37 patients (70%) after 3 months of CRT. Sensitivity and specificity of either shuffle or multiphasic septal motion for all 5 observers (range 90% to 97% and 67% to 83%, respectively) were found to predict LV dyssynchrony. To predict LV reverse remodeling, sensitivity and specificity from 87% to 92% and 69% to 81% were observed, respectively. In conclusion, the qualitative observation of a typical shuffle or multiphasic septal motion predicts LV dyssynchrony and LV reverse remodeling adequately.

This paper presents a predictive-coding algorithm for the compression of multiple depth-sequences obtained from a multi-camera acquisition setup. The proposed depth-prediction algorithm works by synthesizing a virtual depth-image that matches the depth-image (of the predicted camera). To generate this virtual depth-image, we use an image-rendering algorithm known as 3D image-warping. This newly proposed prediction technique is employed in a 3D coding system in order to compress multiview depth-sequences. For this purpose, we introduce an extended H.264 encoder that employs two prediction techniques: a block-based motion prediction and the previously mentioned 3D image-warping prediction. This extended H.264 encoder adaptively selects the most efficient prediction scheme for each image-block using a rate-distortion criterion. We present experimental results for several multiview depth-sequences, which show a quality improvement of about 2.5 dB as compared to H.264 inter-coded depth-images.

Most existing feature-point matching algorithms rely on photometric region descriptors to distinct and match feature points in two images. In this paper, we propose an efficient feature-point matching algorithm for finding point correspondences between two uncalibrated images separated by small or mid camera baselines. The proposed algorithm does not rely on photometric descriptors for matching. Instead, only the motion smoothness constraint is used, which states that the correspondence vectors within a small neighborhood usually have similar directions and magnitudes. The correspondences of feature points in a neighborhood are collectively determined in such a way that the smoothness of the local correspondence field is maximized. The smoothness constraint is self-contained in the correspondence field and is robust to the camera motion, scene structure, illumination, etc. This makes the entire point-matching process texture-independent, descriptor-free and robust. The experimental results show that the proposed method performs much better than the intensity-based block-matching technique, even when the image contrast varies clearly across images.

The quality of health care depends, among other factors, on the quality of a physician's domain knowledge. Since it is impossible to keep up with all new findings and developments, physicians usually have gaps in their domain knowledge. To handle exceptional cases, access to the full range of medical literature is required. The specific literature needed for appropriate treatment of the patient is described by a physician's information need. Physicians are often unaware of their information needs. To support them, this paper presents a first step towards automatically formulating patient-related information needs. We start investigating how we can model a physician's information needs in general. Then we propose an approach to instantiate the model into a representation of a physician's information needs using the patient data as stored in a medical record. Our experiments show that this approach is feasible. Since the number of formulated patient-related information needs is rather high, it has to be reduced. To reduce the number of formulated information needs we propose the use of additional knowledge. Four types of knowledge are discussed, viz. (a) knowledge about temporal aspects, (b) domain knowledge, (c) knowledge about a physician's specialism, and (d) a user model. Future research has to clarify which type of knowledge (or combination thereof) is most appropriate for our purpose. It is expected that the resultant set of information needs will have a manageable size and contributes to the quality of health care.

A main stream contains successive content elements of video and/or audio information that encode video and/or audio information at a first data rate. A computation circuit (144) computes main fingerprints from the successive content elements. A reference stream is received having a second data rate lower than the first data rate. The reference stream defines a sequence of the reference fingerprints. A comparator unit (144) compares the main fingerprints with the reference fingerprints. The main stream is monitored for the presence of inserted content elements between original content elements, where the original content elements have main fingerprints that match successive reference fingerprints and the inserted content elements have main fingerprints that do not match reference fingerprints. Rendering of inserted content elements to be skipped.; In an embodiment when more than one content element matches only one is rendered. In another embodiment matching is used to control zapping to or from the main stream. In another embodiment matching is used to control linking of separately received mark-up information such as subtitles to points in the main stream.

This paper investigates the performance of M-ary amplitude-phase shift keying (APSK) digital modulation over typical nonlinear satellite channels. The effect of the satellite nonlinearity is studied, and distortion pre- and post-compensation techniques for coded APSK are presented. Moreover, clock timing, signal amplitude and carrier phase recovery schemes are discussed. For the latter, a new class of non turbo decoder-aided closed-loop phase synchronizers featuring good performance and low complexity is studied. Finally, an end-to-end coded APSK system simulator inclusive of the satellite channel model and synchronization sub-systems is discussed and its performance compared to standard trellis-coded QAM concatenated with Reed-Solomon codes, showing a remarkable gain in both power and spectral efficiency. Coded APSK, recently selected for the new standard -DVB-S2- for digital video broadcasting and interactive broadband satellite services, is shown to represent a powerand spectral-efficient solution for satellite nonlinear channels

With the growing storage capacities of hard-disks and optical discs, large consumer video databases are gradually developing. The large effective storage capacity using compressed video leads to the application of fast storage and retrieval functions, to enable quick user-friendly searching for and access to specific parts of the video data. We explore the feasibility of a near real-time semantic sports video analyzer for an experimental consumer media server. This tool is able to automatically extract and analyze key events in a video sequence. The analyzer employs several visual cues and a model for real-world coordinates, so that key parameters (e.g. speed and position) of a player can be determined with sufficient accuracy. This special data can be stored as metadata, thereby facilitating intelligent searching of events. The tool consists of four processing steps: (1) playing frame detection, (2) court extraction, as well as a 3-D camera model, (3) player segmentation and tracking, and (4) event-based high-level analysis exploiting visual cues extracted in the real-world. Our system has been evaluated in a new distributed AV content analysis system for home entertainment. We show attractive experimental results indicating the system efficiency and classification skills, thereby offering new analysis and search/retrieval tools to the consumer

Object-oriented coding in the MPEG-4 standard enables the separate processing of foreground objects and the scene background (sprite). Since the background sprite only has to be sent once, transmission bandwidth can be saved. We have found that the counter-intuitive approach of splitting the background into several independent parts can reduce the overall amount of data. Furthermore, we show that in the general case, the synthesis of a single background sprite is even impossible and that the scene background must be sent as multiple sprites instead. For this reason, we propose an algorithm that provides an optimal partitioning of a video sequence into independent background sprites (a multisprite), resulting in a significant reduction of the involved coding cost. Additionally, our sprite-generation algorithm ensures that the sprite resolution is kept high enough to preserve all details of the input sequence, which is a problem especially during camera zoom-in operations. Even though our sprite generation algorithm creates multiple sprites instead of only a single background sprite, it is fully compatible with the existing MPEG-4 standard. The algorithm has been evaluated with several test sequences, including the well-known Table-tennis and Stefan sequences. The total coding cost for the sprite VOP is reduced by a factor of about 2.6 or even higher, depending on the sequence.

Large adaptive filters are frequently used in diverse applications such as channel equalization, interference suppression, beamforming, etc. The least mean squared (LMS) algorithm and its variants form one of the basic building blocks of adaptive systems. The frequency domain implementations of the LMS algorithm are preferred in practice since the computational burden of LMS can be reduced significantly by using the vast family of fast fourier transform (FFT) algorithms. Despite the advantage of frequency domain LMS over regular-LMS, the increasing computational complexity of the FFT-based LMS algorithms (with filter length) makes them unattractive for applications with a large number of filter taps. In this paper, FFT pruning is used to reduce the computational cost of frequency domain LMS by exploiting the structure of the LMS algorithm. We study various pruning strategies with our objective being reduction in computational burden and conclude that transform decomposition is the most appropriate pruning strategy. Using this pruning technique, worst-case computational savings of 10% and 5% can be achieved for applications that use filter lengths on the order of a few hundreds and a few thousands, respectively. In delay-sensitive applications, substantially more savings can be effected by pruning. � 2006 Elsevier B.V. All rights reserved.

OBJECTIVE: The optimal use of pulsed wave Doppler tissue imaging (DTI) in predicting left ventricular (LV) reverse remodeling after cardiac resynchronization therapy (CRT) was investigated. METHODS: DTI was performed in 69 patients before and 3 months after CRT. Echocardiographic reverse remodeling was observed in 38 patients. LV dyssynchrony was measured with the time to onset or peak systolic velocity in 2- and 6-basal segment models. RESULTS: The time to onset and either the standard deviation of 6 segments of > 20 ms or a delay of > or = 60 ms between any 2 of 6 segments had a similar predictive accuracy (sensitivity, 97% and 95%, respectively; specificity, 74% and 73%, respectively). The time to peak systolic velocity or evaluating 2 segments was less accurate. CONCLUSIONS: Evaluation of 6 segments is necessary to predict LV reverse remodeling after CRT. The time to onset of systolic velocity is superior to the time to peak velocity.

In high-density data storage systems, noise becomes highly correlated and data dependent as a result of media noise, channel nonlinearities, and front-end filters. In such environments, conventional timing recovery schemes will exhibit large residual timing jitter and, especially, data-dependent timing jitter. This paper presents a new data-aided timing recovery algorithm for data storage systems with data-dependent noise. We derive a maximum-likelihood timing recovery scheme based on a data-dependent Gauss-Markov model of the noise. The timing recovery algorithm incorporates data-dependent noise prediction parameters in the form of linear prediction filters and prediction error variances. Moreover, because noise can be nonstationary in practice, we propose an adaptive algorithm to estimate and track the noise prediction parameters. Simulation results, for an idealized optical storage channel incorporating a simple model of media noise, illustrate the merits of our algorithm

This paper investigates the design of power and spectrally efficient coded modulations based on amplitude phase shift keying (APSK) modulation with application to satellite broadband communications. APSK represents an attractive modulation format for digital transmission over nonlinear satellite channels due to its power and spectral efficiency combined with its inherent robustness against nonlinear distortion. For these reasons APSK has been very recently introduced in the new standard for satellite Digital Video Broadcasting named DVB-S2. Assuming an ideal rectangular transmission pulse, for which no nonlinear inter-symbol interference is present and perfect pre-compensation of the nonlinearity, we optimize the APSK constellation. In addition to the minimum distance criterion, we introduce a new optimization based on the mutual information; this new method generates an optimum constellation for each spectral efficiency. To achieve power efficiency jointly with low bit error rate (BER) floor we adopt a powerful binary serially concatenated turbo-code coupled with optimal APSK modulations through bit-interleaved coded modulation. We derive tight approximations on the maximum-likelihood decoding error probability, and results are compared with computer simulations. The proposed coded modulation scheme is shown to provide a considerable performance advantage compared to current standards for satellite multimedia and broadcasting systems.

The least mean squared (LMS) algorithm and its variants have been the most often used algorithm in adaptive signal processing. However the LMS algorithm suffers from a high computational complexity, especially with large filter lengths. The Fourier transform-based block normalized LMS (FBNLMS) reduces the computation count by using the discrete Fourier transform (DFT) and exploiting the fast algorithms for implementing the DFT. Even though the savings achieved with the FBNLMS over the direct-LMS implementation are significant, the computational requirements of FBNLMS are still very high, rendering many real-time applications, like audio and video estimation, infeasible. The Hartley transform-based BNLMS (HBNLMS) is found to have a computational complexity much less than, and a memory requirement almost of the same order as, that of the FBNLMS. This paper is based on the cosine and sine symmetric implementation of the discrete Hartley transform (DHT), which is the key in reducing the computational complexity of the FBNLMS by 33% asymptotically (with respect to multiplications). The parallel implementation of the discrete cosine transform (DCT) in turn can lead to more efficient implementations of the HBNLMS. ? 2005 IEEE.

Dataflow analysis techniques are key to reduce the number of design iterations and shorten the design time of real-time embedded network based multiprocessor systems that process data streams. With these analysis techniques the worst-case end-to-end temporal behavior of hard real-time applications can be derived from a dataflow model in which computation, communication and arbitration is modeled. For soft real-time applications these static dataflow analysis techniques are combined with simulation of the dataflow model to test statistical assertions about their temporal behavior. The simulation results in combination with properties of the dataflow model are used to derive the sensitivity of design parameters and to estimate parameters like the capacity of data buffers.

Complexity scalability offers the advantage of one-time design of video applications for a large product family, including mobile devices, without the need of redesigning the applications on the algorithmic level to meet the requirements of the different products. In this paper, we present complexity scalable MPEG encoding having core modules with modifications for scalability. The interdependencies of the scalable modules and the system performance are evaluated. Experimental results show scalability giving a smooth change in complexity and corresponding video quality. Scalability is basically achieved by varying the number of computed DCT coefficients and the number of evaluated motion vectors but other modules are designed such they scale with the previous parameters. In the experiments using the “Stefan” sequence, the elapsed execution time of the scalable encoder, reflecting the computational complexity, can be gradually reduced to roughly 50% of its original execution time. The video quality scales between 20 dB and 48 dB PSNR with unity quantizer setting, and between 21.5 dB and 38.5 dB PSNR for different sequences targeting 1500 kbps. The implemented encoder and the scalability techniques can be successfully applied in mobile systems based on MPEG video compression.

Complexity scalability algorithms are important for mobile consumer devices using MPEG video coding, because they offer a trade-off between picture quality and the embedded available computational performance. This paper presents a new scalable three-stage motion estimation technique, which includes preprocessing of frames in display order and approximation of motion-vector fields using multiple temporal references. A quality refinement of the approximation is added as an optional stage. Furthermore, we present a new scalable motion-estimation algorithm, based on simple edge detection, for integration into the above-mentioned new three-stage motion estimation technique. The complete system provides a flexible framework with a large scalability range in computational effort, resulting in an output quality that scales up smoothly with the number of operations spent on the motion estimation process. Experiments show a scalable computational effort from below one SAD (sum of absolute difference) computation per macroblock up to 15 SAD computations, resulting in a global variation of 7.4 dB PSNR in picture quality (with the "Stefan" sequence). In high-quality operation, the new algorithm is comparable to (or even outperforms) a full-search motion estimation with a search window of 32/spl times/32 pixels.

Blind signal separation can easily find its position in audio applications where mutually independent sources need to be separated from their microphone mixtures while both room acoustics and sources are unknown. However, the conventional separation algorithms can hardly be implemented in real time due to the high computational complexity. The computational load is mainly caused by either direct or indirect estimation of thousands of acoustic parameters. Aiming at the complexity reduction, in this paper, the acoustic paths are investigated through an acoustic similarity index (ASI). Then a new mixing model is proposed. With closely spaced microphones (5–10 cm apart), the model relieves the computational load of the separation algorithm by reducing the number and length of the filters to be adjusted. To cope with real situations, a blind audio signal separation algorithm (BLASS) is developed on the proposed model. BLASS only uses the second-order statistics (SOS) and performs efficiently in frequency domain.

The detection of road lane markings has many practical applications, such as advanced driver assistance systems and road maintenance. In this paper we propose an algorithm to detect and recognize road lane markings from panoramic images. Our algorithm consists of four steps. First, an inverse perspective mapping is applied to the image, and the potential road markings are segmented based on their intensity difference compared to the surrounding pixels. Second, we extract the distance between the center and the boundary at regular angular steps of each considered potential road marking segment into a feature vector. Third, each segment is classified using a Support Vector Machine (SVM). Finally, by modeling the lane markings, previous false positive detected segments can be rejected based on their orientation and position relative to the lane markings. Our experiments show that the system is capable of recognizing \(93\,\%\), \(95\,\%\) and \(91\,\%\) of striped line segments, blocks and arrows respectively, as well as \(94\,\%\) of the lane markings.

This paper addresses the issue of separating multiple speakers from mixtures of these that are obtained using multiple microphones in a room. An adaptive blind signal separation algorithm, which is entirely based on second-order statistics, is derived. One of the advantages of this algorithm is that no parameters need to be tuned. Moreover, an extension of the algorithm that can simultaneously deal with blind signal separation and echo cancellation is derived. Experiments with real recordings have been carried out, showing the effectiveness of the algorithm for real-world signals.

Eclipse is a scalable architecture template for designing data-dependent stream-processing subsystems of media-processing SoCs. It combines application configuration flexibility with the efficiency of function-specific coprocessors that concurrently execute the tasks of one or more applications.

In this paper, a 6.7-kbps vector sum excited linear prediction (VSELP) coder with less computational complexity is presented. A very efficient VSELP codebook with nine basis vectors and a heuristic K-selection method (to reduce the search space and complexity) is constructed to obtain the stochastic codebook vector. The nine basis vectors are obtained by optimizing a set of randomly generated basis vectors. During the optimization process, we have trained the basis vectors to give the system apriori knowledge of the characteristics of the input. The coder is implemented on a TMS320C541 digital signal processor. The performance is evaluated by testing the 6.7-kbps VSELP coder with different test speech data taken from different speakers. The quality of the coder is estimated by comparing the performance of the 6.7-kbps VSELP coder with an 8-kbps VSELP speech coder based on the IS-54 standards. ? 2002 Elsevier Science B.V. All rights reserved.

This paper describes a novel model for training an event detection system based on object tracking. We propose to model the training as a multiple instance learning problem, which allows us to train the classifier from annotated events despite temporal ambiguities. We apply this technique to realize a Perimeter Intrusion Detection (PID) algorithm and employ image-based features to distinguish real objects from moving vegetation and other distractions. An earlier developed tracking system is extended with the proposed technique to create an on-line PID-event detection system. Experiments with challenging videos show a reduction of the number of false positives by a factor 2–3 and improve the F1 detection performance from 0.15 to 0.28, when compared to a commercially available PID algorithm.

A surround sound system comprises a receiver (301) for receiving a multichannel spatial signal that comprises at least one surround channel. A directional ultrasound transducer (305) is used for emitting ultrasound towards a surface to reach a listening position (111) via a reflection of the surface. The ultrasound signal may specifically reach the listening position from the side, above or behind of a nominal listener. A first drive unit (303) generates a drive signal for the directional ultrasound transducer (301) from the surround channel. The use of an ultrasound transducer for providing the surround sound signal provides an improved spatial experience while allowing the speaker to be located e.g. to the front of the user.; In particular, an ultrasound beam is much narrower and well defined than conventional audio beams and can accordingly better be directed to provide the desired reflections. In some scenarios, the ultrasound transducer (305) may be supplemented by an audio range loudspeaker (309).

An apparatus comprises a test signal generator (401) which generates an ultrasonic test signal by modulating an audio band test signal on an ultrasonic signal. The ultrasonic test signal is radiated from a parametric loudspeaker (403) and is demodulated by non-linearities in the air. A reflected audio signal may arise from reflections of an object, such as a wall. An audio band sensor (405) generates an audio band captured signal which comprises the demodulated reflected audio band signal. A distance circuit (407) then generates a distance estimate for the distance from the parametric loudspeaker (403) to the object in response to a comparison of the audio band captured signal and the audio band test signal. Specifically two signals may be correlated to determine a delay corresponding to the full path length.; Based on the distance estimates an audio environment may be estimated and a sound system may be adapted accordingly.

Long LMS filters of the tapped-delay line type are in widespread use, particularly in acoustic applications. For the limiting case of an infinite line length it is shown that the behavior of such filters is governed by remarkably simple laws. This is true for the steady state, where for small stepsizes the weight-error correlations become independent of the input signal, but also for the transient behavior, where the spatial Fourier transform of the weight-error distribution decays exponentially. Moreover, a necessary and (probably) sufficient stability bound for the stepsize is derived. The "wave theory" developed for the infinite line length also predicts the behavior of rather short filters with sufficient accuracy, particularly for a moderately colored input signal. No independence assumption is required and no assumption concerning the spectral distribution of the additive noise. Under steady-state conditions, the weight-error correlation between two line taps is solely determined by the noise autocorrelation, with the time delay replaced by the tap distance.

Advanced broadcast manipulation of TV sequences and enhanced user interfaces for TV systems have resulted in an increased amount of pre- and post-editing of video sequences, where graphical information is inserted. However, in the current broadcasting chain, there are no provisions for enabling an efficient transmission/storage of these mixed video and graphics signals and, at this emerging stage of DTV systems, introducing new standards is not desired. Nevertheless, in the professional video communication chain between content provider and broadcaster and locally, in the DTV receiver, proprietary video-graphics compression schemes can be used to enable more efficient transmission/storage of mixed video and graphics signals. For example, in the DTV receiver case this will lead to a significant memory-cost reduction. To preserve a high overall image quality, the video and graphics data require independent coding systems, matched with their specific visual and statistical properties. We introduce various efficient algorithms that support both the lossless (contour, runlength and arithmetic coding) and the lossy (block predictive coding) compression of graphics data. If the graphics data are a-priori mixed with video and the graphics position is unknown at compression time, an accurate detection mechanism is applied to distinguish the two signals, such that independent coding algorithms can be employed for each data-type. In the DTV memory-reduction scenario, an overall bit-rate control completes the system, ensuring a fixed compression factor of 2-3 per frame without sacrificing the quality of the graphics.

A new video processing architecture for high-end TV applications is presented, featuring a flexible heterogeneous multi-processor architecture, executing video tasks in parallel and independently. The signal flow graph and the processors are programmable, enabling an optimal picture quality for different TV display modes. The concept is verified by an experimental chip design. The architecture allows several video streams to be processed and displayed in parallel and in a programmable way, with an individual signal quality.

This paper presents a generic multiprocessor architecture for video processing, featuring an array of programmable application-specific coprocessors and a programmable switch-bar communication network. Key advantages of the platform are that the order of functions can be changed and that video tasks can be programmed at system and functional levels using tasks of any length, by applying dynamic data-flow. This modular concept is cost-effective and suitable for a large range of applications and features in the video domain. The system can be well applied in combination with microcontrollers or media processors and it is used for implementing an experimental TV platform.