Search

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.

In large-scale automatic traffic sign surveying systems, the primary computational effort is concentrated at the traffic sign detection stage. This paper focuses on reducing the computational load of particularly the sliding window object detection algorithm which is employed for traffic sign detection. Sliding-window object detectors often use a linear SVM to classify the features in a window. In this case, the classification can be seen as a convolution of the feature maps with the SVM kernel. It is well known that convolution can be efficiently implemented in the frequency domain, for kernels larger than a certain size. We show that by careful reordering of sliding-window operations, most of the frequency-domain transformations can be eliminated, leading to a substantial increase in efficiency. Additionally, we suggest to use the overlap-add method to keep the memory use within reasonable bounds. This allows us to keep all the transformed kernels in memory, thereby eliminating even more domain transformations, and allows all scales in a multiscale pyramid to be processed using the same set of transformed kernels. For a typical sliding-window implementation, we have found that the detector execution performance improves with a factor of 5.3. As a bonus, many of the detector improvements from literature, e.g. chi-squared kernel approximations, sub-class splitting algorithms etc., can be more easily applied at a lower performance penalty because of an improved scalability.

A single stage blind source extraction (BSE) approach to extract one desired source signal from an instantaneous mixture of several signals is studied in this paper. In contrast to methods based on blind source separation (BSS) followed by a classifier we extract directly the desired source signal. In recent papers it has been shown that prior information about the desired source's mixing column or autocorrelation function can be incorporated in a single stage BSE algorithm. The focus of this paper is to give insight in the BSE problem for scenarios where joint diagonalization of correlation matrices is applicable, i.e., to exploit the full potential of the SOS of observed signals. In this work we unify and expand these previously presented BSE methods, which lead to the unified instantaneous blind source extraction (UIBSE) algorithm. We show that UIBSE is even more flexible than previously presented since both types of prior information may be combined to obtain better results in selecting the desired source. Above that, the same algorithm may be used for different objective functions. Finally, a performance comparison with BSS algorithms that are followed by a classifier shows that UIBSE is more robust to noisy observations.

We present an approach for efficient rendering and transmitting views to a high-resolution autostereoscopic display for medical purposes. Displaying biomedical images on an autostereoscopic display poses different requirements than in a consumer case. For medical usage, it is essential that the perceived image represents the actual clinical data and offers sufficiently high quality for diagnosis or understanding. Autostereoscopic display of multiple views introduces two hurdles: transmission of multi-view data through a bandwidth-limited channel and the computation time of the volume rendering algorithm. We address both issues by generating and transmitting limited set of views enhanced with a depth signal per view. We propose an efficient view interpolation and rendering algorithm at the receiver side based on texture+depth data representation, which can operate with a limited amount of views. We study the main artifacts that occur during rendering-occlusions, and we quantify them first for a synthetic model and then for real-world biomedical data. The experimental results allow us to quantify the peak signal-to-noise ratio for rendered texture and depth as well as the amount of disoccluded pixels as a function of the angle between surrounding cameras.

Current development of 3D technologies brings 3DTV within reach for the customers. We discuss in this article the recent advancements in free-viewpoint interpolation for 3D video. This technology is still a research topic and many efforts are dedicated to creation, evaluation and improvement of new algorithms. First, we describe a recent DIBR algorithm that allows free-viewpoint interpolation. We perform detailed analysis of its performance and establish the sources of errors that occur in the resulting 3D videos. Our graph presents each DIBR step’s contribution to the overall error. We show that the challenge consists of finding the correct pixel values in the areas of resulting image where warping did not provide any information. The quality of the resulting image is mostly determined by the warping procedure and cannot be considerably changed by post-processing. To accommodate the generation of stereoscopic videos, we discuss the possibilities of creating a free-viewpoint stereoscopic video.

Multi-view 3D video may succeed stereo 3DTV in multimedia and TV applications. The MPEG committee has installed a special task force to establish a standard for multiview 3D video coding. Enabling free-viewpoint selection by the consumer is one of the major purposes of multi-view 3D encoded data. One focal point of our research work concerns an efficient implementation of the rendering part of stereoscopic free-viewpoint visualization, which is a computationally expensive task and significantly determines the final image quality. The free-viewpoint DIBR algorithm that is described in this article is implemented with an off-theshelf GPU that can be integrated in advanced 3DTV systems. We present the principal steps of a representative freeviewpoint DIBR, based on image warping and image artifact reduction techniques, and show the key differences between the reference software and our GPU implementation. One of those differences is the joint execution of signal processing blocks to share memory usage. Using a combination of the highly parallel programming architecture CUDA and a graphics API, we have achieved real-time performance executing 1080p HD multi-view video with a rendering quality that is comparable to the software implementation.

The term smart environment refers to physical spaces equipped with sensors feeding into adaptive algorithms that enable the environment to become sensitive and responsive to the presence and needs of its occupants. People with special needs, such as the elderly or disabled people, stand to benefit most from such environments as they offer sophisticated assistive functionalities supporting independent living and improved safety. In a smart environment, the key issue is to sense the location and identity of its users. In this paper, we intend to tackle the problems of detecting and tracking humans in a realistic home environment by exploiting the complementary nature of (synchronized) color and depth images produced by a low-cost consumer-level RGB-D camera. Our system selectively feeds the complementary data emanating from the two vision sensors to different algorithmic modules which together implement three sequential components: (1) object labeling based on depth data clustering, (2) human re-entry identification based on comparing visual signatures extracted from the color (RGB) information, and (3) human tracking based on the fusion of both depth and RGB data. Experimental results show that this division of labor improves the system?s efficiency and classification performance.

This paper presents an algorithm for localization and classification of subtitles in TV videos. We extend an existing static-region detector with object-based adaptive temporal filtering, bounding box computation around blobs of refined static regions, bounding box categorization based on geometry and filling degree of static regions, and subtitle classification using text-stroke alignment features. On a test set of more than 5000 video frames, a Precision rate of 96% is achieved at 98% Recall rate. The system detects subtitles without frame delays, and uses techniques suitable for implementation in a TV platform. We also experimentally show that the picture quality of Motion-Compensated Picture Rate Conversion in televisions can benefit from our system.

Watching sports, such as tennis and soccer, remains popular for a broad class of consumers. However, audiences might not be able to enjoy their favorite games from a television or PC when they are traveling, so mobile devices are increasingly used for watching sports video. Unfortunately, watching sports on a mobile device is not as simple as watching sports on TV. Bandwidth limitations on wireless networks prevent high bit-rate video transmission. In addition, small displays lose visual details of the sports event. Bandwidth limitations occur primarily when multiple users want to stream video on the same wireless link. These bottlenecks are likely to occur because of the popularity of an event. This article describes a camera modeling based, mixed-reality system concept. The idea is to build a 3D model for sports video, where all parameters of this model can be obtained by analyzing the broadcasting video. Instead of sending original images to the mobile device, the system only sends parameters of the 3D model and the information about players and balls, which can significantly save transmission bandwidth. Additionally, because we have full-quality information about camera modeling and also players and balls, the mobile client is able to recover the important information without loss of the visual details. In addition, we can generate virtual scenes for less important areas, such as the playing field and the commercial billboard, without changing the major story of the sports game.

In this paper, we consider the problem of finding and localizing social human groups in videos, which can form a basis for further analysis and monitoring of groups in general. Our approach is motivated by the collective behavior of individuals which has a fundament in sociological studies. We design a detection-based multi-target tracking framework which is capable of handling short-term occlusions and producing stable trajectories. Human groups are discovered by clustering trajectories of individuals in an agglomerative fashion. A novel similarity function related to distances between group members, robustly measures the similarity of noisy trajectories. We have evaluated our approach on several test sequences and achieved acceptable miss rates (19.4%, 29.7% and 46.7%) at reasonable false positive detections per frame (0.129, 0.813 and 0.371). The relatively high miss rates are caused by a strict evaluation procedure, whereas the visual results are quite acceptable.

Brain-computer interface (BCI) systems based on the steady-state visual evoked potential (SSVEP) provide higher information throughput and require shorter training than BCI systems using other brain signals. To elicit an SSVEP, a repetitive visual stimulus (RVS) has to be presented to the user. The RVS can be rendered on a computer screen by alternating graphical patterns, or with external light sources able to emit modulated light. The properties of an RVS (e.g., frequency, color) depend on the rendering device and influence the SSVEP characteristics. This affects the BCI information throughput and the levels of user safety and comfort. Literature on SSVEP-based BCIs does not generally provide reasons for the selection of the used rendering devices or RVS properties. In this paper, we review the literature on SSVEP-based BCIs and comprehensively report on the different RVS choices in terms of rendering devices, properties, and their potential influence on BCI performance, user safety and comfort.

A fundamental challenge limiting information quality obtained from smart sensing garments is the influence of textile movement relative to limbs. We present and validate a comprehensive modeling and simulation framework to predict recognition performance in casual loose-fitting garments. A statistical posture and wrinkle-modeling approach is introduced to simulate sensor orientation errors pertained to local garment wrinkles. A metric was derived to assess fitting, the body-garment mobility. We validated our approach by analyzing simulations of shoulder and elbow rehabilitation postures with respect to experimental data using actual casual garments. Results confirmed congruent performance trends with estimation errors below 4% for all study participants. Our approach allows to estimate the impact of fitting before implementing a garment and performing evaluation studies with it. These simulations revealed critical design parameters for garment prototyping, related to performed body posture, utilized sensing modalities, and garment fitting. We concluded that our modeling approach can substantially expedite design and development of smart garments through early-stage performance analysis.

This paper aims at generating an automated way to evaluate the team-behavior of trainees in a delivery simulation course using video-processing techniques with emphasis on multiple people tracking. The paper is composed of two interacting, but clearly separated stages: moving people segmentation and multiple people tracking. At people segmentation stage, the combination of the Gaussian Mixture Model (GMM) and the Dynamic Markov Random Fields (DMRF) technique helps to extract the foreground pixels. For a better extraction of the human silhouettes, the energy function of DMRF is extended with texture information. At multiple people tracking stage, we concentrate on solving human-occlusion problem caused by interacting persons based on silhouette data and a non-linear regression model. Our model effectively transfers the person location problem during the occlusion into the finding of the local maximum points on a smooth curve, so that visual persons in the partial or complete occlusion can still be precisely captured. We have compared our algorithm with two other popular tracking algorithms: mean-shift and particle-filter. Experimental results reveal that the correctness of our method is much higher than the mean-shift algorithm and slightly lower than a particle-filter, however, with the major benefit of being a factor of 10---15 faster in computing.

This paper addresses privacy leakage in biometric secrecy systems. Four settings are investigated. The first one is the standard Ahlswede-Csiszar secret-generation setting in which two terminals observe two correlated sequences. They form a common secret by interchanging a public message. This message should only contain a negligible amount of information about the secret, but here, in addition, we require it to leak as little information as possible about the biometric data. For this first case, the fundamental tradeoff between secret-key and privacy-leakage rates is determined. Also for the second setting, in which the secret is not generated but independently chosen, the fundamental secret-key versus privacy-leakage rate balance is found. Settings three and four focus on zero-leakage systems. Here the public message should only contain a negligible amount of information on both the secret and the biometric sequence. To achieve this, a private key is needed, which can only be observed by the terminals. For both the generated-secret and the chosen-secret model, the regions of achievable secret-key versus private-key rate pairs are determined. For all four settings, the fundamental balance is determined for both unconditional and conditional privacy leakage.

This paper addresses the automatic analysis of court-net sports video content. We extract information about the players, the playing-field in a bottom-up way until we reach scene-level semantic concepts. Each part of our framework is general, so that the system is applicable to several kinds of sports. A central point in our framework is a camera calibration module that relates the a-priori information of the geometric layout in the form of a court model to the input image. Exploiting this information, several novel algorithms are proposed, including playing-frame detection, players segmentation and tracking. To address the player-occlusion problem, we model the contour map of the player silhouettes using a nonlinear regression algorithm, which enables to locate the players during the occlusions caused by players in the same team. Additionally, a Bayesian-based classifier helps to recognize predefined key events, where the input is a number of real-world visual features. We illustrate the performance and efficiency of the proposed system by evaluating it for a variety of sports videos containing badminton, tennis and volleyball, and we show that our algorithm can operate with more than 91% feature detection accuracy and 90% event detection.

Background Laparoscopic surgery is normally performed under general anaesthesia, but regional techniques have been found beneficial, usually in the management of patients with major medical problems. Encouraged by such experience, we performed a feasibility study of segmental spinal anaesthesia in healthy patients. Methods Twenty ASA I or II patients undergoing elective laparoscopic cholecystectomy received a segmental (T10 injection) spinal anaesthetic using 1 ml of bupivacaine 5 mg ml −1 mixed with 0.5 ml of sufentanil 5 µg ml −1 . Other drugs were only given (systemically) to manage patient anxiety, pain, nausea, hypotension, or pruritus during or after surgery. The patients were reviewed 3 days postoperatively by telephone. Results The spinal anaesthetic was performed easily in all patients, although one complained of paraesthesiae which responded to slight needle withdrawal. The block was effective for surgery in all 20 patients, six experiencing some discomfort which was readily treated with small doses of fentanyl, but none requiring conversion to general anaesthesia. Two patients required midazolam for anxiety and two ephedrine for hypotension. Recovery was uneventful and without sequelae, only three patients (all for surgical reasons) not being discharged home on the day of operation. Conclusions This preliminary study has shown that segmental spinal anaesthesia can be used successfully and effectively for laparoscopic surgery in healthy patients. However, the use of an anaesthetic technique involving needle insertion into the vertebral canal above the level of termination of the spinal cord requires great caution and should be restricted in application until much larger numbers of patients have been studied.

A multiphasic septal motion and typical septal-to-lateral apical shuffle of the left ventricle can be observed echocardiographically in some patients with left branch bundle block. The relation of both with left ventricular (LV) dyssynchrony according to tissue Doppler and LV reverse remodeling after cardiac resynchronization therapy was investigated. Fifty-three patients (37 men; age 68 ± 8 years) with ischemic (n = 26) or idiopathic (n = 27) cardiomyopathy, baseline QRS duration 171 ± 30 ms, LV ejection fraction 21 ± 7%, and LV end-diastolic volume 257 ± 91 ml were studied. LV dyssynchrony using tissue Doppler was considered present if the SD of the interval between QRS and onset of systolic velocity of 6 basal LV segments was >20 ms. Shuffle was evaluated visually independently by 5 cardiologists and considered present if observed in =1 view. LV reverse remodeling, defined as LV end-systolic volume decrease =10%, was observed in 37 patients (70%) after 3 months of CRT. Sensitivity and specificity of either shuffle or multiphasic septal motion for all 5 observers (range 90% to 97% and 67% to 83%, respectively) were found to predict LV dyssynchrony. To predict LV reverse remodeling, sensitivity and specificity from 87% to 92% and 69% to 81% were observed, respectively. In conclusion, the qualitative observation of a typical shuffle or multiphasic septal motion predicts LV dyssynchrony and LV reverse remodeling adequately.

This paper presents a predictive-coding algorithm for the compression of multiple depth-sequences obtained from a multi-camera acquisition setup. The proposed depth-prediction algorithm works by synthesizing a virtual depth-image that matches the depth-image (of the predicted camera). To generate this virtual depth-image, we use an image-rendering algorithm known as 3D image-warping. This newly proposed prediction technique is employed in a 3D coding system in order to compress multiview depth-sequences. For this purpose, we introduce an extended H.264 encoder that employs two prediction techniques: a block-based motion prediction and the previously mentioned 3D image-warping prediction. This extended H.264 encoder adaptively selects the most efficient prediction scheme for each image-block using a rate-distortion criterion. We present experimental results for several multiview depth-sequences, which show a quality improvement of about 2.5 dB as compared to H.264 inter-coded depth-images.

Most existing feature-point matching algorithms rely on photometric region descriptors to distinct and match feature points in two images. In this paper, we propose an efficient feature-point matching algorithm for finding point correspondences between two uncalibrated images separated by small or mid camera baselines. The proposed algorithm does not rely on photometric descriptors for matching. Instead, only the motion smoothness constraint is used, which states that the correspondence vectors within a small neighborhood usually have similar directions and magnitudes. The correspondences of feature points in a neighborhood are collectively determined in such a way that the smoothness of the local correspondence field is maximized. The smoothness constraint is self-contained in the correspondence field and is robust to the camera motion, scene structure, illumination, etc. This makes the entire point-matching process texture-independent, descriptor-free and robust. The experimental results show that the proposed method performs much better than the intensity-based block-matching technique, even when the image contrast varies clearly across images.

The quality of health care depends, among other factors, on the quality of a physician's domain knowledge. Since it is impossible to keep up with all new findings and developments, physicians usually have gaps in their domain knowledge. To handle exceptional cases, access to the full range of medical literature is required. The specific literature needed for appropriate treatment of the patient is described by a physician's information need. Physicians are often unaware of their information needs. To support them, this paper presents a first step towards automatically formulating patient-related information needs. We start investigating how we can model a physician's information needs in general. Then we propose an approach to instantiate the model into a representation of a physician's information needs using the patient data as stored in a medical record. Our experiments show that this approach is feasible. Since the number of formulated patient-related information needs is rather high, it has to be reduced. To reduce the number of formulated information needs we propose the use of additional knowledge. Four types of knowledge are discussed, viz. (a) knowledge about temporal aspects, (b) domain knowledge, (c) knowledge about a physician's specialism, and (d) a user model. Future research has to clarify which type of knowledge (or combination thereof) is most appropriate for our purpose. It is expected that the resultant set of information needs will have a manageable size and contributes to the quality of health care.

With the growing storage capacities of hard-disks and optical discs, large consumer video databases are gradually developing. The large effective storage capacity using compressed video leads to the application of fast storage and retrieval functions, to enable quick user-friendly searching for and access to specific parts of the video data. We explore the feasibility of a near real-time semantic sports video analyzer for an experimental consumer media server. This tool is able to automatically extract and analyze key events in a video sequence. The analyzer employs several visual cues and a model for real-world coordinates, so that key parameters (e.g. speed and position) of a player can be determined with sufficient accuracy. This special data can be stored as metadata, thereby facilitating intelligent searching of events. The tool consists of four processing steps: (1) playing frame detection, (2) court extraction, as well as a 3-D camera model, (3) player segmentation and tracking, and (4) event-based high-level analysis exploiting visual cues extracted in the real-world. Our system has been evaluated in a new distributed AV content analysis system for home entertainment. We show attractive experimental results indicating the system efficiency and classification skills, thereby offering new analysis and search/retrieval tools to the consumer

Object-oriented coding in the MPEG-4 standard enables the separate processing of foreground objects and the scene background (sprite). Since the background sprite only has to be sent once, transmission bandwidth can be saved. We have found that the counter-intuitive approach of splitting the background into several independent parts can reduce the overall amount of data. Furthermore, we show that in the general case, the synthesis of a single background sprite is even impossible and that the scene background must be sent as multiple sprites instead. For this reason, we propose an algorithm that provides an optimal partitioning of a video sequence into independent background sprites (a multisprite), resulting in a significant reduction of the involved coding cost. Additionally, our sprite-generation algorithm ensures that the sprite resolution is kept high enough to preserve all details of the input sequence, which is a problem especially during camera zoom-in operations. Even though our sprite generation algorithm creates multiple sprites instead of only a single background sprite, it is fully compatible with the existing MPEG-4 standard. The algorithm has been evaluated with several test sequences, including the well-known Table-tennis and Stefan sequences. The total coding cost for the sprite VOP is reduced by a factor of about 2.6 or even higher, depending on the sequence.

OBJECTIVE: The optimal use of pulsed wave Doppler tissue imaging (DTI) in predicting left ventricular (LV) reverse remodeling after cardiac resynchronization therapy (CRT) was investigated. METHODS: DTI was performed in 69 patients before and 3 months after CRT. Echocardiographic reverse remodeling was observed in 38 patients. LV dyssynchrony was measured with the time to onset or peak systolic velocity in 2- and 6-basal segment models. RESULTS: The time to onset and either the standard deviation of 6 segments of > 20 ms or a delay of > or = 60 ms between any 2 of 6 segments had a similar predictive accuracy (sensitivity, 97% and 95%, respectively; specificity, 74% and 73%, respectively). The time to peak systolic velocity or evaluating 2 segments was less accurate. CONCLUSIONS: Evaluation of 6 segments is necessary to predict LV reverse remodeling after CRT. The time to onset of systolic velocity is superior to the time to peak velocity.

In high-density data storage systems, noise becomes highly correlated and data dependent as a result of media noise, channel nonlinearities, and front-end filters. In such environments, conventional timing recovery schemes will exhibit large residual timing jitter and, especially, data-dependent timing jitter. This paper presents a new data-aided timing recovery algorithm for data storage systems with data-dependent noise. We derive a maximum-likelihood timing recovery scheme based on a data-dependent Gauss-Markov model of the noise. The timing recovery algorithm incorporates data-dependent noise prediction parameters in the form of linear prediction filters and prediction error variances. Moreover, because noise can be nonstationary in practice, we propose an adaptive algorithm to estimate and track the noise prediction parameters. Simulation results, for an idealized optical storage channel incorporating a simple model of media noise, illustrate the merits of our algorithm

This paper investigates the design of power and spectrally efficient coded modulations based on amplitude phase shift keying (APSK) modulation with application to satellite broadband communications. APSK represents an attractive modulation format for digital transmission over nonlinear satellite channels due to its power and spectral efficiency combined with its inherent robustness against nonlinear distortion. For these reasons APSK has been very recently introduced in the new standard for satellite Digital Video Broadcasting named DVB-S2. Assuming an ideal rectangular transmission pulse, for which no nonlinear inter-symbol interference is present and perfect pre-compensation of the nonlinearity, we optimize the APSK constellation. In addition to the minimum distance criterion, we introduce a new optimization based on the mutual information; this new method generates an optimum constellation for each spectral efficiency. To achieve power efficiency jointly with low bit error rate (BER) floor we adopt a powerful binary serially concatenated turbo-code coupled with optimal APSK modulations through bit-interleaved coded modulation. We derive tight approximations on the maximum-likelihood decoding error probability, and results are compared with computer simulations. The proposed coded modulation scheme is shown to provide a considerable performance advantage compared to current standards for satellite multimedia and broadcasting systems.

Complexity scalability offers the advantage of one-time design of video applications for a large product family, including mobile devices, without the need of redesigning the applications on the algorithmic level to meet the requirements of the different products. In this paper, we present complexity scalable MPEG encoding having core modules with modifications for scalability. The interdependencies of the scalable modules and the system performance are evaluated. Experimental results show scalability giving a smooth change in complexity and corresponding video quality. Scalability is basically achieved by varying the number of computed DCT coefficients and the number of evaluated motion vectors but other modules are designed such they scale with the previous parameters. In the experiments using the “Stefan” sequence, the elapsed execution time of the scalable encoder, reflecting the computational complexity, can be gradually reduced to roughly 50% of its original execution time. The video quality scales between 20 dB and 48 dB PSNR with unity quantizer setting, and between 21.5 dB and 38.5 dB PSNR for different sequences targeting 1500 kbps. The implemented encoder and the scalability techniques can be successfully applied in mobile systems based on MPEG video compression.

The detection of road lane markings has many practical applications, such as advanced driver assistance systems and road maintenance. In this paper we propose an algorithm to detect and recognize road lane markings from panoramic images. Our algorithm consists of four steps. First, an inverse perspective mapping is applied to the image, and the potential road markings are segmented based on their intensity difference compared to the surrounding pixels. Second, we extract the distance between the center and the boundary at regular angular steps of each considered potential road marking segment into a feature vector. Third, each segment is classified using a Support Vector Machine (SVM). Finally, by modeling the lane markings, previous false positive detected segments can be rejected based on their orientation and position relative to the lane markings. Our experiments show that the system is capable of recognizing \(93\,\%\), \(95\,\%\) and \(91\,\%\) of striped line segments, blocks and arrows respectively, as well as \(94\,\%\) of the lane markings.

Eclipse is a scalable architecture template for designing data-dependent stream-processing subsystems of media-processing SoCs. It combines application configuration flexibility with the efficiency of function-specific coprocessors that concurrently execute the tasks of one or more applications.

This paper describes a novel model for training an event detection system based on object tracking. We propose to model the training as a multiple instance learning problem, which allows us to train the classifier from annotated events despite temporal ambiguities. We apply this technique to realize a Perimeter Intrusion Detection (PID) algorithm and employ image-based features to distinguish real objects from moving vegetation and other distractions. An earlier developed tracking system is extended with the proposed technique to create an on-line PID-event detection system. Experiments with challenging videos show a reduction of the number of false positives by a factor 2–3 and improve the F1 detection performance from 0.15 to 0.28, when compared to a commercially available PID algorithm.

Advanced broadcast manipulation of TV sequences and enhanced user interfaces for TV systems have resulted in an increased amount of pre- and post-editing of video sequences, where graphical information is inserted. However, in the current broadcasting chain, there are no provisions for enabling an efficient transmission/storage of these mixed video and graphics signals and, at this emerging stage of DTV systems, introducing new standards is not desired. Nevertheless, in the professional video communication chain between content provider and broadcaster and locally, in the DTV receiver, proprietary video-graphics compression schemes can be used to enable more efficient transmission/storage of mixed video and graphics signals. For example, in the DTV receiver case this will lead to a significant memory-cost reduction. To preserve a high overall image quality, the video and graphics data require independent coding systems, matched with their specific visual and statistical properties. We introduce various efficient algorithms that support both the lossless (contour, runlength and arithmetic coding) and the lossy (block predictive coding) compression of graphics data. If the graphics data are a-priori mixed with video and the graphics position is unknown at compression time, an accurate detection mechanism is applied to distinguish the two signals, such that independent coding algorithms can be employed for each data-type. In the DTV memory-reduction scenario, an overall bit-rate control completes the system, ensuring a fixed compression factor of 2-3 per frame without sacrificing the quality of the graphics.

A new video processing architecture for high-end TV applications is presented, featuring a flexible heterogeneous multi-processor architecture, executing video tasks in parallel and independently. The signal flow graph and the processors are programmable, enabling an optimal picture quality for different TV display modes. The concept is verified by an experimental chip design. The architecture allows several video streams to be processed and displayed in parallel and in a programmable way, with an individual signal quality.

This paper presents a generic multiprocessor architecture for video processing, featuring an array of programmable application-specific coprocessors and a programmable switch-bar communication network. Key advantages of the platform are that the order of functions can be changed and that video tasks can be programmed at system and functional levels using tasks of any length, by applying dynamic data-flow. This modular concept is cost-effective and suitable for a large range of applications and features in the video domain. The system can be well applied in combination with microcontrollers or media processors and it is used for implementing an experimental TV platform.

Accurate and up-to-date inventories of lighting poles are of interest to energy companies, beneficial for the transition to energy-efficient lighting and may contribute to a more adequate lighting of streets. This potentially improves social security and reduces crime and vandalism during nighttime. This paper describes a system for automated surveying of lighting poles from street-level panoramic images. The system consists of two independent detectors, focusing at the detection of the pole itself and at the detection of a specific lighting fixture type. Both follow the same approach, and start with detection of the feature of interest (pole or fixture) within the individual images, followed by a multi-view analysis to retrieve the real-world coordinates of the poles. Afterwards, the detection output of both algorithms is merged. Large-scale validations, covering about 135 km of road, show that over 91% of the lighting poles is found, while the precision remains above 50%. When applying this system in a semi-automated fashion, high-quality inventories can be created up to 5 times more efficiently compared to manually surveying all poles from the images.

This paper focuses on road sign classification for creating accurate and up-to-date inventories of traffic signs, which is important for road safety and maintenance. This is a challenging multi-class classification task, as a large number of different sign types exist which only differ in minor details. Moreover, changes in viewpoint, capturing conditions and partial occlusions result in large intra-class variations. Ideally, road sign classification systems should be robust against these variations, while having an acceptable computational load. This paper presents a classification approach based on the popular Bag Of Words (BOW) framework, which we optimize towards the best trade-off between performance and execution time. We analyze the performance aspects of PCA-based dimensionality reduction, soft and hard assignment for BOW codebook matching and the codebook size. Furthermore, we provide an efficient implementation scheme. We compare these techniques to design a fast and accurate BOW-based classification scheme. This approach allows for the selection of a fast but accurate classification methodology. This BOW approach is compared against structural classification, and we show that their combination outperforms both individual methods. This combination, exploiting both BOW and structural information, attains high classification scores (96.25% to 98%) on our challenging real-world datasets.

This paper presents an automatic ship detection approach for video-based port surveillance systems. Our approach combines context and motion saliency analysis. The context is represented by the assumption that ships only travel inside a water region. We perform motion saliency analysis since we expect ships to move with higher speed compared to the water flow and static environment. A robust water detection is first employed to extract the water region as contextual information in the video frame, which is achieved by graph-based segmentation and region-based classification. After the water detection, the segments labeled as non-water are merged to form the regions containing candidate ships, based on the spatial adjacency. Finally, ships are detected by checking motion saliency for each candidate ship according to a set of criteria. Experiments are carried out with real-life surveillance videos, where the obtained results prove the accuracy and robustness of the proposed ship detection approach. The proposed algorithm outperforms a state-of-the-art algorithm when applied to the same sets of surveillance videos. © (2013) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

This paper evaluates a case study where a previously reported texture-adaptive skin detection algorithm is applied for TV image enhancement. A color-only skin detector of an existing high-end TV chip is extended with a texture feature, enabling exclusion of skin-colored textured areas. We report the performance in terms of detection result, and in terms of image quality in a cascade of three image enhancement functions. In terms of detection score, at 80% true positive rate, the false positive rate of the texture-adaptive skin detector is 29% lower than that of the color-only skin detector, forming a clear improvement. With respect to its application in enhancement, we assess the enhancement quality by measuring the RMS error of the enhancement output compared to an optimally enhanced image based on ground-truth skin areas. When using the texture-adaptive skin detector, the enhancement RMS error is 44% lower than the RMS error when using the color-only skin detector, thereby confirming the applicability of the proposal. Subjective evaluation indicates that the proposed algorithm is better suitable for mid/high-frequency boosting applications like sharpness enhancement, and less suitable for enhancements that operate on low frequencies like color correction functions 1.

Inventories of traffic signs are acquired from street-level images in a semi-automated fashion, employing object detection and classification techniques. This is a challenging task, as signs are captured from different viewpoints and under various weather conditions. Furthermore, many similar signs exist, only differing in minor details, and moreover, sign-like objects occur frequently. Consequently, current state-of-the-art systems are unable to reach the required quality level, implying the need for manual corrections. This involves checking all classification results to correct the small minority of misclassifications. This paper presents a classification approach aiming at both high recognition scores and predicting the reliability of the classification output, enabling selective manual intervention. Two reliability prediction methods are compared, analyzing either the classifier scores, or matching the input samples with predefined templates. Large-scale experiments performed for three sign classes, each containing numerous sign types, show that over 80% of the correctly classified results can be marked as reliable, while not marking any misclassifications as reliable. Hence, our research shows that a reliable prediction is possible and that manual invention can be concentrated to the about 25% remaining samples only. Overall, 92.7% of the 8, 159 signs are classified correctly.

Free-ViewPoint (FVP) interpolation allows creating a new view between the existing reference views. Applied to 3D multi-view video sequences, it leads to two important applications: (1) FVP service provided to the user, which enables the possibility to interactively select the viewing point of the scene; (2) improved compression of multi-view video sequences by using view prediction for inter-view coding. In this chapter, the authors provide an overview of the essential steps for 3D free-view video communication, which consists of the free-viewpoint interpolation techniques, a concept for free-view coding and a scalable free-view video streaming architecture. For facilitating free-view to the user, the chapter introduces the free-viewpoint interpolation techniques and the concept of warping. The authors assume that 3D video is represented by texture and depth images available for each view. Therefore it is possible to apply Depth Image Based Rendering (DIBR), which uses the depth signal as a important cue for geometry information and 3D reconstruction. Authors analyze the involved interpolation problems, such as cracks, ghost contours and disocclusions, which arise from an FVP interpolation and propose several solutions to improve the image quality of the synthesized view. Afterwards, they present a standard approach to FVP rendering used currently by the research community and our FVP interpolation. Additionally, authors show the use of FVP rendering for the multi-view coding and streaming and discuss the gains and trade-offs of it. At the end of the chapter are the state-of-the-art achievements and challenges of FVP rendering and a vision concerning the development of free-viewpoint services.

This paper presents a skin detection algorithm for TV applications, and investigates the feasibility of its real-time implementation on a DSP and an FPGA platform. We present the details of the algorithm, which combines a flexible histogram-based color feature, and a color-constrained texture feature suitable for skin detection. First we show the improvement in detection result, as compared to existing color-only detectors. Then we explore three implementations: (1) full-precision non real-time SW on DSP, (2) reduced-precision real-time SW on DSP, and (3) full-precision realtime hardware on FPGA. We conclude that even though a real-time SW implementation is possible, its quality is sub-optimal and the solution is too expensive for a practical consumer TV system. Therefore, hardware support is indispensable for full-precision realtime execution of the proposed algorithm on HD video.

We developed a digital signal processor (DSP) platform that includes a box equipped with the C5505 eZdsp and easy to use software tools. This box includes a TMS320C5505 DSP, an audio interface, several communication interfaces, and a USB programming interface. Example applications show that the DSP platform is easy to use. This platform is developed by the Signal Processing Systems group at Eindhoven University of Technology and is used in DSP projects and education.

This paper considers large scale traffic sign detection on a dataset consisting of high-resolution street-level panoramic photographs. Traffic signs are automatically detected and classified with a set of state-of-the-art algorithms. We introduce a color transformation to extend a Histogram of Oriented Gradients (HOG) based detection algorithm to further improve the performance. This transformation uses a specific set of reference colors that aligns with traffic sign characteristics, and measures the distance of each pixel to these reference colors. This results in an improved consistency on the gradients at the outer edge of the traffic sign. In an experiment with 33, 400 panoramic images, the number of misdetections decreased by 53.6% and 51.4% for red/blue circular signs, and by 19.6% and 28.4% for yellow speed bump signs, measured at a realistic detector operating point.

Traffic sign inventories are important to governmental agencies as they facilitate evaluation of traffic sign locations and are beneficial for road and sign maintenance. These inventories can be created (semi-)automatically based on street-level panoramic images. In these images, object detection is employed to detect the signs in each image, followed by a classification stage to retrieve the specific sign type. Classification of traffic signs is a complicated matter, since sign types are very similar with only minor differences within the sign, a high number of different signs is involved and multiple distortions occur, including variations in capturing conditions, occlusions, viewpoints and sign deformations. Therefore, we propose a method for robust classification of traffic signs, based on the Bag of Words approach for generic object classification. We extend the approach with a flexible, modular codebook to model the specific features of each sign type independently, in order to emphasize at the inter-sign differences instead of the parts common for all sign types. Additionally, this allows us to model and label the present false detections. Furthermore, analysis of the classification output provides the unreliable results. This classification system has been extensively tested for three different sign classes, covering 60 different sign types in total. These three data sets contain the sign detection results on street-level panoramic images, extracted from a country-wide database. The introduction of the modular codebook shows a significant improvement for all three sets, where the system is able to classify about 98% of the reliable results correctly.

Single-microphone speech enhancement algorithms that employ trained codebooks of parametric representations of speech spectra have been shown to be successful in the suppression of non-stationary noise, e.g., in mobile phones. In this paper, we introduce the concept of a context-dependent codebook, and look at two aspects of context: dependency on the particular speaker using the mobile device, and on the acoustic condition during usage (e.g., hands-free mode in a reverberant room). Such context-dependent codebooks may be trained on-line. A new scheme is proposed to appropriately combine the estimates resulting from the context-dependent and context-independent codebooks under a Bayesian framework. Experimental results establish that the proposed approach performs better than the context-independent codebook in the case of a context match and better than the context-dependent codebook in the case of a context mismatch.

Loudspeakers are often modelled as a rigid piston in an infinite baffle. As a model for real loudspeakers, this approach is limited in two ways. One issue is that a loudspeaker cone is not rigid, and a second issue is that a loudspeaker is mostly used in a cabinet. Both issues are addressed here by developing the velocity of the radiator in terms of orthogonal polynomials known from optical diffraction theory as Zernike circle polynomials. Using these polynomials we develop semi-analytic expressions for the sound pressure from the radiator in two different cases: as a flexible flat radiator mounted in an infinite baffle, and as the cap of a rigid sphere. In the latter case the comparison is done not only for the pressure but also for other quantities viz. the baffle-step response, sound power and directivity, and the acoustic centre of the radiator. These quantities are compared with those from a real loudspeaker. Finally, in the case of the baffled-piston radiation the spatial impulse response is presented.

Accurate inventories of traffic signs are required for road maintenance and increase of the road safety. These inventories can be performed efficiently based on street-level panoramic images. However, this is a challenging problem, as these images are captured under a wide range of weather conditions. Besides this, occlusions and sign deformations occur and many sign look-a-like objects exist. Our approach is based on detecting present signs in panoramic images, both to derive a classification code and to combine multiple detections into an accurate position of the signs. It starts with detecting the present signs in each panoramic image. Then, all detections are classified to obtain the specific sign type, where also false detections are identified. Afterwards, detections from multiple images are combined to calculate the sign positions. The performance of this approach is extensively evaluated in a large, geographical region, where over 85% of the 3; 341 signs are automatically localized, with only 3:2% false detections. As nearly all missed signs are detected in at least a single image, only very limited manual interactions have to be supplied to safeguard the performance for highly accurate inventories.

The availability of large-scale databases containing street-level panoramic images offers the possibility to perform semi-automatic surveying of real-world objects such as traffic signs. These inventories can be performed significantly more efficiently than using conventional methods. Governmental agencies are interested in these inventories for maintenance and safety reasons. This paper introduces a complete semi-automatic traffic sign inventory system. The system consists of several components. First, a detection algorithm locates the 2D position of the traffic signs in the panoramic images. Second, a classification algorithm is used to identify the traffic sign. Third, the 3D position of the traffic sign is calculated using the GPS position of the photographs. Finally, the results are listed in a table for quick inspection and are also visualized in a web browser.

Text detection and recognition in natural images have conventionally been seen in the prior art as autonomous tasks executed in a strictly sequential processing chain with limited information sharing between sub-systems. This approach is flawed because it introduces (1) redundancy in extracting the same text properties multiple times and (2) error by prohibiting verification of hard (often binarized) detection results at later stages. We explore the possibilities for integration of detection and recognition modules by a feedforward multidimensional information stream. Integration involves suitable characterization of the text string at detection and application of the knowledge to ease recognition by a given OCR system. The choice of characterization properties generally depends on the OCR system, although some of them have proven universally applicable. We show that the proposed integration measures enable more robust recognition of text in complex, unconstrained natural environments. Specifically, integration by the proposed measures (1) eliminates textual input irregularities that recognition engines cannot handle and (2) adaptively tunes the recognition stage for each input image. The former function boosts correct detections, while the latter mainly reduces the number of false positives. Our validation experiments on a set of low-quality natural images show that adaptively tuning the OCR stage to the typical text-to-background transitions in the input image (gradient significance profiling) allows to attain an improvement of 29% in the precision-recall performance, mostly through boosting precision.

Prematurely born infants are observed in a Neonatal Intensive Care Unit (NICU) for medical treatment. These infants are nursed in an incubator, where their vital body functions such as heart rate, respiration, blood pressure, oxygen saturation, and temperature are continuously monitored. However, the existing monitoring system is lack of the measurement for visual expression of the neonatal. Therefore, valuable information about the well being of the patient (e.g., pain and discomfort) may pass unnoticed. This chapter aims at designing a prototype of an automated video monitoring system for the detection of discomfort in newborns by analyzing their facial expression. The system consists of several algorithmic components, ranging from the face detection, ROI determination, facial feature extraction, to behavior stage classification. To further adapt this system to the real hospital environment, the authors also intend to address the problem of locating the face regions under varying lighting conditions. To this end, an adaptive face detection technique based on gamut mapping is presented. The authors have evaluated the prototype system on recordings of a healthy newborn with different conditions, and we show that our algorithm can operate with approximately 88% accuracy.

Recently we have shown how a blind source extraction (BSE) algorithm can be equipped with some prior information about mixing parameters of the desired source in order to extract this source. The prior information, which may contain errors, is used to construct a matrix from linear combinations of correlation matrices. The extraction filter is easily obtained from the specific eigenstructure of this matrix. Here we project the beamformer design problem onto the above mentioned BSE algorithm by parameterizing the mixing system. We show in three ways that the proposed method is efficient and flexible. First, with one procedure an LCMV and MVDR beamformer can be obtained. Second, by taking only two appropriate linear combinations of correlation matrices, which may be interpreted as selection beamformers, the desired source can be selected. Third, selection beamformers can be designed for a subset of sensors while the final beamformer exploits data from all sensors.

Traffic sign inventories are created for road safety and maintenance based on street-level panoramic images. Due to the large capturing interval, large viewpoint deviations between the different capturings occur. These viewpoint variations complicate the classification procedure, which aims at the selection of the correct sign type, out of a high number of nearly similar sign types, typically resulting in misclassifications. This paper describes a novel approach for incorporating viewpoint information to the classification procedure, where the sign orientation is estimated based on dense matching. Afterwards, each sample is corrected to a frontal viewpoint, which is then classified. Finally, the sign type is obtained by weighted voting. Large-scale experiments including 2, 224 traffic signs show that this approach reduces the misclassification rate by about 33% compared to the single-view case.