Search

Modern methods for counting people in crowded scenes rely on deep networks to estimate people densities in individual images. As such, only very few take advantage of temporal consistency in video sequences, and those that do only impose weak smoothness constraints across consecutive frames. In this paper, we advocate estimating people flows across image locations between consecutive images and inferring the people densities from these flows instead of directly regressing them. This enables us to impose much stronger constraints encoding the conservation of the number of people. As a result, it significantly boosts performance without requiring a more complex architecture. Furthermore, it allows us to exploit the correlation between people flow and optical flow to further improve the results. We also show that leveraging people conservation constraints in both a spatial and temporal manner makes it possible to train a deep crowd counting model in an active learning setting with much fewer annotations. This significantly reduces the annotation cost while still leading to similar performance to the full supervision case. [ABSTRACT FROM AUTHOR]

Land cover (LC) and LC change information are essential data in terrestrial surface research. However, the LC products are highly inconsistent, especially in the mountainous area. Most LC assessment studies focused on the consistency of spatial patterns, while ignoring the consistency of change and elevation heterogeneity. In this article, four LC products were assessed for spatiotemporal consistency on the Tibetan Plateau (TP), including the moderate resolution imaging spectroradiometer LC (MCD12Q1), the climate change initiative LC (CCI-LC), the 30-meter global land cover (Globeland30) and the multiperiod land use/ LC remote sensing monitoring database in China. The impact of elevation on the consistency of multiple LC products across spatiotemporal scales was further analyzed. The spatial consistency of the three and more products was about 70%, with higher consistency for grassland and bare land and lower consistency for wetland and shrubland on the TP. Globeland30 and CCI-LC were better than others for overall monitoring, with the inconsistency of less than 45% by Google Earth dataset validation. The temporal change consistency of the four LC products was less than 10%. With increasing elevation, the average spatial consistency decreased and the LC change area and temporal change consistency increased. There is a high inconsistency of LC changes on the TP in existing commonly used products, demonstrating the need to develop high-quality LC products in long time series.

As an essential vegetation structural parameter, leaf area index (LAI) is involved in many critical biochemical processes, such as photosynthesis, respiration, and precipitation interception. The MODerate resolution Imaging Spectroradiometer (MODIS) and Visible Infrared Imager Radiometer Suite (VIIRS) LAI sequence products have long supported various global climate, biogeochemistry, and energy flux research. These applications all rely on the accuracy of the product's long time series. However, uncontrolled interferences (e.g., adverse observation conditions and sensor uncertainties) potentially introduce substantial uncertainties to time series in product applications. As one of the most sensitive areas in response to global climate change, the Tibet Plateau (TP) has been treated as a crucial testing ground for thousands of studies on vegetation. To ensure the credibility of the studies arising from MODIS/VIIRS LAI products, the temporal quality uncertainties of data need to be clarified. This article proposed a method to revisit the temporal stability of the MODIS (MOD and MYD) and VIIRS (VNP) LAI in the TP, expecting to provide useful information for better accounting for the uncertainties in this area. Results show that the MODIS and VIIRS LAI were relatively stable in time series and available to be used continuously, among which the temporal quality of the MODIS LAI was the most stable. Moreover, the MODIS and VIIRS LAI products performed similarly in both time-series stability and time-series anomaly distribution, magnitudes and fluctuations. The time-series stability evaluation strategy applied to the MODIS and VIIRS LAI can also be employed to other remote sensing products.

Dynamic artistic text style transfer aims to migrate the style in terms of both the appearance and motion patterns from a reference style video to the target text to create artistic text animation. Recent researches have improved the usability of transfer models by introducing texture control. However, it remains an important open challenge to investigate the control of the stylistic degree with respect to shape deformation. In this paper, we explore a new problem of dynamic artistic text style transfer with glyph stylistic degree control. The key idea is to build multi-scale glyph-style shape mappings through a novel bidirectional shape matching framework. Following this idea, we first introduce a scale-ware Shape-Matching GAN to learn such mappings to simultaneously model the style shape features at multiple scales and transfer them onto the target glyph. Furthermore, an advanced Shape-Matching GAN++ is proposed to animate a static text image based on the reference style video. Our Shape-Matching GAN++ characterizes the short-term consistency of motion patterns via shape matchings within consecutive frames, which are propagated to achieve effective long-term consistency. Experiments show that the proposed method outperforms previous state-of-the-arts both qualitatively and quantitatively, and generate high-quality and controllable artistic text. [ABSTRACT FROM AUTHOR]

Video-based human pose estimation (VHPE) is a vital yet challenging task. While deep learning algorithms have made tremendous progress for the VHPE, lots of these approaches to this task implicitly model the long-range interaction between joints by expanding the receptive field of the convolution or designing a graph manually. Unlike prior methods, we design a lightweight and plug-and-play joint relation extractor (JRE) to explicitly and automatically model the associative relationship between joints. The JRE takes the pseudo heatmaps of joints as input and calculates their similarity. In this way, the JRE can flexibly learn the correlation between any two joints, allowing it to learn the rich spatial configuration of human poses. Furthermore, the JRE can infer invisible joints according to the correlation between joints, which is beneficial for locating occluded joints. Then, combined with temporal semantic continuity modeling, we propose a Relation-based Pose Semantics Transfer Network (RPSTN) for video-based human pose estimation. Specifically, to capture the temporal dynamics of poses, the pose semantic information of the current frame is transferred to the next with a joint relation guided pose semantics propagator (JRPSP). The JRPSP can transfer the pose semantic features from the non-occluded frame to the occluded frame. The proposed RPSTN achieves state-of-the-art or competitive results on the video-based Penn Action, Sub-JHMDB, PoseTrack2018, and HiEve datasets. Moreover, the proposed JRE improves the performance of backbones on the image-based COCO2017 dataset. Code is available at https://github.com/YHDang/pose-estimation. [ABSTRACT FROM AUTHOR]

Deep convolutional neural network based video super-resolution (SR) models have achieved significant progress in recent years. Existing deep video SR methods usually impose optical flow to wrap the neighboring frames for temporal alignment. However, accurate estimation of optical flow is quite difficult, which tends to produce artifacts in the super-resolved results. To address this problem, we propose a novel end-to-end deep convolutional network that dynamically generates the spatially adaptive filters for the alignment, which are constituted by the local spatio-temporal channels of each pixel. Our method avoids generating explicit motion compensation and utilizes spatio-temporal adaptive filters to achieve the operation of alignment, which effectively fuses the multi-frame information and improves the temporal consistency of the video. Capitalizing on the proposed adaptive filter, we develop a reconstruction network and take the aligned frames as input to restore the high-resolution frames. In addition, we employ residual modules embedded with channel attention as the basic unit to extract more informative features for video SR. Both quantitative and qualitative evaluation results on three public video datasets demonstrate that the proposed method performs favorably against state-of-the-art super-resolution methods in terms of clearness and texture details. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new 4D segmentation formulation, aiming to improve the temporal consistency of adults' brain tissue segmentation. In our method, tissue segmentation result is represented by the membership functions of the tissues, which are derived as a result of minimizing energy in a fuzzy C-means (FCM) framework. We first introduce a variational formulation with a temporal consistency constraint on the membership functions, then convert the constraint to a vector-valued function from which the membership functions are directly computed according to an analytically defined mapping. These vector-valued functions capture the bias field and intensity means of each tissue addressing the temporal variation in intensity inhomogeneities and the intensity means of the images. The effectiveness is demonstrated on the Baltimore Longitudinal Study of Aging (BLSA) benchmark dataset. Our method takes few parameters which are easy to tune and achieves 96.4% of TC factor for gray matter, 98.1% of TC factor for white matter, significantly superior to the compared methods on this research line.

Video deblurring is still a challenging low-level vision task since spatio-temporal characteristics across both the spatial and temporal domains are difficult to model. In this article, to model the temporal information, we develop a non-local block which estimates inter-frame similarity and inter-frame difference. Specially, for modeling the spatial characteristics and restoring sharp frame details, we propose a recursive block that iteratively refines feature maps generated at the last iteration. In addition, a novel temporal loss function is introduced to ensure the temporal consistency of generated frames. Experimental results on public datasets demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

3D dynamic point clouds provide a natural discrete representation of real-world objects or scenes in motion, with a wide range of applications in immersive telepresence, autonomous driving, surveillance, etc. Nevertheless, dynamic point clouds are often perturbed by noise due to hardware, software or other causes. While a plethora of methods have been proposed for static point cloud denoising, few efforts are made for the denoising of dynamic point clouds, which is quite challenging due to the irregular sampling patterns both spatially and temporally. In this paper, we represent dynamic point clouds naturally on spatial-temporal graphs, and exploit the temporal consistency with respect to the underlying surface (manifold). In particular, we define a manifold-to-manifold distance and its discrete counterpart on graphs to measure the variation-based intrinsic distance between surface patches in the temporal domain, provided that graph operators are discrete counterparts of functionals on Riemannian manifolds. Then, we construct the spatial-temporal graph connectivity between corresponding surface patches based on the temporal distance and between points in adjacent patches in the spatial domain. Leveraging the initial graph representation, we formulate dynamic point cloud denoising as the joint optimization of the desired point cloud and underlying graph representation, regularized by both spatial smoothness and temporal consistency. We reformulate the optimization and present an efficient algorithm. Experimental results show that the proposed method significantly outperforms independent denoising of each frame from state-of-the-art static point cloud denoising approaches, on both Gaussian noise and simulated LiDAR noise. [ABSTRACT FROM AUTHOR]

The model drift problem is an inevitable problem for online visual tracking. Model drift can amplify the false tracking result over time and lead to failed tracking finally. Background-aware correlation filter (BACF) tracker obtains the training samples from negative background patches instead of shifted foreground patches and can mitigate the drifting to some extent. However, in complex scenes, such as occlusion, deformation, the inappropriate update of BACF may lead to model drift. We propose an anti-drift background-aware correlation filter via introducing the temporal consistency constraint into BACF. That is, we combine the excellent ability of BACF to distinguish the foreground and background with the ability of the temporal consistency constraint to stabilize model changes, and improve anti-drift performance of BACF. To improve computation efficient, we offer a fast algorithm via alternative Direction Method of Multipliers (ADMM) in the frequency domain. Besides, we design a simple yet effective adaptive feature channel selection method, which can further improve the success rate and precision of the tracker in complex scenes. Our proposed tracker with hand-crafted features achieves a gain of 2.2%, 3.3%, and 5.3% in AUC success rate on OTB-2013, OTB-2015, and Temple Color-128 dataset, respectively. Our proposed tracker with deep features achieves AUC success rate of 69.1% on OTB-2013 dataset. Besides, our proposed tracker with hand-crafted features achieves a gain of 3.94% in EAO on VOT-2018 dataset. Moreover, the proposed tracker performs favorably against several representative state-of-the-art methods regarding precision and success rate.

In recent years, neural style transfer has attracted more and more attention, especially for image style transfer. However, temporally consistent style transfer for videos is still a challenging problem. Existing methods, either relying on a significant amount of video data with optical flows or using single-frame regularizers, fail to handle strong motions or complex variations, therefore have limited performance on real videos. In this article, we address the problem by jointly considering the intrinsic properties of stylization and temporal consistency. We first identify the cause of the conflict between style transfer and temporal consistency, and propose to reconcile this contradiction by relaxing the objective function, so as to make the stylization loss term more robust to motions. Through relaxation, style transfer is more robust to inter-frame variation without degrading the subjective effect. Then, we provide a novel formulation and understanding of temporal consistency. Based on the formulation, we analyze the drawbacks of existing training strategies and derive a new regularization. We show by experiments that the proposed regularization can better balance the spatial and temporal performance. Based on relaxation and regularization, we design a zero-shot video style transfer framework. Moreover, for better feature migration, we introduce a new module to dynamically adjust inter-channel distributions. Quantitative and qualitative results demonstrate the superiority of our method over state-of-the-art style transfer methods. Our project is publicly available at: https://daooshee.github.io/ReReVST/. [ABSTRACT FROM AUTHOR]

Video super-resolution (SR) aims at generating a sequence of high-resolution (HR) frames with plausible and temporally consistent details from their low-resolution (LR) counterparts. The key challenge for video SR lies in the effective exploitation of temporal dependency between consecutive frames. Existing deep learning based methods commonly estimate optical flows between LR frames to provide temporal dependency. However, the resolution conflict between LR optical flows and HR outputs hinders the recovery of fine details. In this paper, we propose an end-to-end video SR network to super-resolve both optical flows and images. Optical flow SR from LR frames provides accurate temporal dependency and ultimately improves video SR performance. Specifically, we first propose an optical flow reconstruction network (OFRnet) to infer HR optical flows in a coarse-to-fine manner. Then, motion compensation is performed using HR optical flows to encode temporal dependency. Finally, compensated LR inputs are fed to a super-resolution network (SRnet) to generate SR results. Extensive experiments have been conducted to demonstrate the effectiveness of HR optical flows for SR performance improvement. Comparative results on the Vid4 and DAVIS-10 datasets show that our network achieves the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Compositing is one of the most important editing operations for images and videos. The process of improving the realism of composite results is often called harmonization. Previous approaches for harmonization mainly focus on images. In this paper, we take one step further to attack the problem of video harmonization. Specifically, we train a convolutional neural network in an adversarial way, exploiting a pixel-wise disharmony discriminator to achieve more realistic harmonized results and introducing a temporal loss to increase temporal consistency between consecutive harmonized frames. Thanks to the pixel-wise disharmony discriminator, we are also able to relieve the need of input foreground masks. Since existing video datasets which have ground-truth foreground masks and optical flows are not sufficiently large, we propose a simple yet efficient method to build up a synthetic dataset supporting supervised training of the proposed adversarial network. The experiments show that training on our synthetic dataset generalizes well to the real-world composite dataset. In addition, our method successfully incorporates temporal consistency during training and achieves more harmonious visual results than previous methods. [ABSTRACT FROM AUTHOR]

Spatiotemporal image fusion is considered as a promising way to provide Earth observations with both high spatial resolution and frequent coverage, and recently, learning-based solutions have been receiving broad attention. However, these algorithms treating spatiotemporal fusion as a single image super-resolution problem, generally suffers from the significant spatial information loss in coarse images, due to the large upscaling factors in real applications. To address this issue, in this paper, we exploit temporal information in fine image sequences and solve the spatiotemporal fusion problem with a two-stream convolutional neural network called StfNet. The novelty of this paper is twofold. First, considering the temporal dependence among image sequences, we incorporate the fine image acquired at the neighboring date to super-resolve the coarse image at the prediction date. In this way, our network predicts a fine image not only from the structural similarity between coarse and fine image pairs but also by exploiting abundant texture information in the available neighboring fine images. Second, instead of estimating each output fine image independently, we consider the temporal relations among time-series images and formulate a temporal constraint. This temporal constraint aiming to guarantee the uniqueness of the fusion result and encourages temporal consistent predictions in learning and thus leads to more realistic final results. We evaluate the performance of the StfNet using two actual data sets of Landsat-Moderate Resolution Imaging Spectroradiometer (MODIS) acquisitions, and both visual and quantitative evaluations demonstrate that our algorithm achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Scheduling real-time update transactions to maintain data freshness is an important issue in real-time database system design, especially for data-intensive embedded applications. Despite years of active study, almost all the past work assumes that the jitter of sensor update transactions can be ignored. But actually, this assumption is not practical in real Cyber-Physical Systems (CPSs), and the existing methods cannot be applied in the design of distributed CPSs. In this paper, we propose, to our best knowledge, the first approach to maintain temporal consistency of real-time data objects for dynamic-priority scheduling when transmission delays are considered. We first propose a method called JB-EDF which calculates periods and deadlines for update transactions with significantly lower workload than that by existing approaches. Next, in order to handle more update transactions, we further propose an enhanced solution called JB-EDF$^*$* which use JB-EDF as a building block. JB-EDF is more suitable for small scale applications, while JB-EDF$^*$* is more attractive for large scale applications. Finally, extensive simulation experiments are conducted. The results demonstrate that JB-EDF and JB-EDF$^*$* outperform existing solutions in terms of both acceptance ratio and system workload, and the proposed methods are also time-efficient. [ABSTRACT FROM AUTHOR]

We propose a novel approach to reconstructing curvilinear tree structures evolving over time, such as road networks in 2D aerial images or neural structures in 3D microscopy stacks acquired in vivo. To enforce temporal consistency, we simultaneously process all images in a sequence, as opposed to reconstructing structures of interest in each image independently. We formulate the problem as a Quadratic Mixed Integer Program and demonstrate the additional robustness that comes from using all available visual clues at once, instead of working frame by frame. Furthermore, when the linear structures undergo local changes over time, our approach automatically detects them. [ABSTRACT FROM AUTHOR]

Millimeter-wave (mmWave) propagation is known to be severely affected by the blockage of the line-of-sight (LoS) path. In contrast to microwave systems, at shorter mmWave wavelengths such blockage can be caused by human bodies, where their mobility within environment makes wireless channel alternate between the blocked and non-blocked LoS states. Following the recent 3GPP requirements on modeling the dynamic blockage as well as the temporal consistency of the channel at mmWave frequencies, in this paper, a new model for predicting the state of a user in the presence of mobile blockers for representative 3GPP scenarios is developed: Urban micro cell street canyon and park/stadium/square. It is demonstrated that the blockage effects produce an alternating renewal process with exponentially distributed non-blocked intervals, and blocked durations that follow the general distribution. The following metrics are derived 1) the mean and the fraction of time spent in blocked/non-blocked state, 2) the residual blocked/non-blocked time, and 3) the time-dependent conditional probability of having blockage/no blockage at time t1 given that there was blockage/no blockage at time t0 . The latter is a function of the arrival rate (intensity), width, and height of moving blockers, distance to the mmWave access point (AP), as well as the heights of the AP and the user device. The proposed model can be used for system-level characterization of mmWave cellular communication systems. For example, the optimal height and the maximum coverage radius of the mmWave APs are derived, while satisfying the required mean data rate constraint. The system-level simulations corroborate that the use of the proposed method considerably reduces the modeling complexity. [ABSTRACT FROM PUBLISHER]

This paper proposes a new hole-filling method with spatiotemporal consistency and binocular symmetry for synthesized 3D videos in view extrapolation. Disocclusion regions in the synthesized views at virtual viewpoints result in regions with missing content. These regions will be referred to as hole regions. To provide the high-quality synthesized 3D videos via 3D display, the hole regions need to be filled considering the characteristics of human visual perception. From the perceptual point of view, binocular asymmetry between synthesized left- and right-eye videos (i.e., stereo pair video) is one of the most important factors that induce visual discomfort in stereoscopic viewing. In addition, binocular symmetry without temporal consistency between temporally neighboring frames could cause visual discomfort by annoying flickering artifacts. In this paper, to maintain the spatiotemporal consistency and binocular symmetry in synthesized 3D videos at multiple virtual viewpoints, we propose a global optimization-based hole-filling method using the information from the already filled adjacent view and previous frame. Furthermore, to reduce the computational cost of the global optimization, we propose a label propagation method, which propagates reliable labels used in the adjacent view and previous frame to the target image to be filled. The performance of the proposed method has been evaluated by objective assessments of 3D image quality, temporal consistency, and computational efficiency. In addition, subjective assessment is conducted for measuring the visual comfort and overall quality. The experimental results proved that the proposed method provides hole filling results with spatiotemporal consistency and binocular symmetry. [ABSTRACT FROM PUBLISHER]

Sparse representation-based methods have been successfully applied to visual tracking. However, complex and inefficient optimization limits their deployment in practical tracking scenarios. In this paper, we propose a temporal consistency dictionary learning tracking algorithm to enable efficient dictionary learning and tracking executive. First, we present an objective function which introduces the fixed dictionary and variance dictionary to reconstruct the object’s appearance. In particular, the proposed method takes the temporal consistency into account by adding a regularization term into the objective function to constrain the difference of object appearance at adjacent frames. Then the optimization problem is solved in an iteration way. Moreover, the proposed method can encode the object’s local structural information, and the local patches from the same candidate altogether for a global appearance representation. Second, we develop an effective observation likelihood function based on the proposed model. It takes the influence of patches with large reconstruction errors into consideration, thereby, alleviating the drifting of the object. Finally, we present an appearance updating strategy to adapt to the object’s appearance variations by the online dictionary learning. Experimental evaluations on the TB50 and TB100 datasets show that the proposed tracking method outperforms sparse representation related visual tracking as well as other state-of-the-art tracking methods. [ABSTRACT FROM AUTHOR]

Maintaining timeliness and data freshness for real-time data objects has long been recognized as an important problem in real-time database research. Despite years of active research, most of the past work focuses on uniprocessor systems. In this paper, we study the workload-aware temporal consistency maintenance problem upon multiprocessor platforms. We consider the problem of how to partition a set of update transactions to m \ge 2 processors to maintain the temporal consistency of real-time data objects under both earliest deadline first (EDF) and deadline monotonic (DM) scheduling in each processor, while minimizing the total workload on m processors. Firstly, we only consider the feasibility aspect of the problem by proposing two polynomial time partitioning schemes, Temporal Consistency Partitioning under EDF (\mathsf TCP\mathsf{EDF} ) and Temporal Consistency Partitioning under DM (\mathsf TCP\mathsf{DM} ), and formally showing that the resource augmentation bounds of both \mathsf TCP\mathsf{EDF} and \mathsf TCP\mathsf{DM} are (3 - \frac1m) . Secondly, we address the partition problem globally by proposing a polynomial time heuristic, Density factor Balancing Fit ( \mathsfDBF ), where density factor balancing plays a major role in producing workload-efficient partitionings. Finally, we evaluate the feasibility and workload performances of \mathsfDBF versus other heuristics with comparable quality experimentally. [ABSTRACT FROM AUTHOR]

In this paper, we propose an efficient method for reliably detecting road lanes based on spatiotemporal images. In an aligned spatiotemporal image generated by accumulating the pixels on a scanline along the time axis and aligning consecutive scanlines, the trajectory of the lane points appears smooth and forms a straight line. The aligned spatiotemporal image is binarized, and two dominant parallel straight lines resulting from the temporal consistency of lane width on a given scanline are detected using a Hough transform, reducing alignment errors. The left and right lane points are then detected near the intersections of the straight lines and the current scanline. Our spatiotemporal domain approach is more robust missing or occluded lanes than existing frame-based approaches. Furthermore, the experimental results show not only computation times reduced to as little as one-third but also a slightly improved rate of detection. [ABSTRACT FROM PUBLISHER]

This paper presents a novel depth video estimation method, which improves estimation property and refines the temporal consistency and spatial accuracy at the same time. The main idea is to incorporate more useful motion and structure information into depth estimation. First, an adaptive weight is calculated based on the motion information of adjacent frames and attached to a temporal term to update the energy function, thus reducing the matching errors and improving temporal consistency. Second, the depth continuity/discontinuity in spatial domain is properly judged by combining the edges of initial depth map and color segmentation, and used in the refinement strategy. Finally, we evaluate the performance of our algorithm with several public stereoscopic video sequences. Experimental results show that the qualities of depth video and synthesized virtual view are all improved. [ABSTRACT FROM PUBLISHER]

Efficient data dissemination is one of the fundamental requirements to enable emerging applications in vehicular cyber–physical systems. In this paper, we present the first study on real-time data services via roadside-to-vehicle communication by considering both the time constraint of data dissemination and the freshness of data items. Passing vehicles can submit their requests to the server, and the server disseminates data items accordingly to serve the vehicles within its coverage. Data items maintained in the database are periodically updated to keep the information up-to-date. We present the system model and analyze challenges on data dissemination by considering both application requirements and communication characteristics. On this basis, we formulate the temporal data dissemination (TDD) problem by introducing the snapshot consistency requirement on serving real-time requests for temporal data items. We prove that TDD is NP-hard by constructing a polynomial-time reduction from the Clique problem. Based on the analysis of the time bound on serving requests, we propose a heuristic scheduling algorithm, which considers the request characteristics of productivity, status, and urgency in scheduling. An extensive performance evaluation demonstrates that the proposed algorithm is able to effectively exploit the broadcast effect, improve the bandwidth efficiency, and enhance the request service chance. [ABSTRACT FROM PUBLISHER]

Deriving deadlines and periods for update transactions so as to maintain timeliness and data freshness while minimizing imposed workload has long been recognized an important problem in real-time database research. Despite years of active research, the state-of-the-art still has much room for improvement, particularly for periodic transactions scheduled by the Earliest Deadline First (EDF) algorithm. In this paper, we propose a practical and efficient two-phase algorithm, GEneral EDF ((\cal G\cal EEDF)), for assigning periods and deadlines to EDF-scheduled update transactions. Phase 1 of (\cal G\cal EEDF) aims at finding solutions for most inputs in linear time, based on the observation that the execution times of update transactions are relatively small compared to the validity interval lengths of real-time data objects in many real-time applications. In the remaining cases for which Phase 1 fails to derive solutions, Phase 2 is invoked by employing an existing deadline-monotonic-based algorithm, which we show is also applicable to our problem. Meanwhile, we have devised several techniques which significantly reduce the cost of schedulability test, and hence greatly improve time efficiency. Our experimental results demonstrate that (\cal G\cal EEDF) outperforms existing approaches in terms of generated workloads. Although Phase 2 has a pseudopolynomial time complexity, our experimental study shows that it runs much faster than other solutions with comparable quality. [ABSTRACT FROM PUBLISHER]

This paper presents a novel approach for depth video enhancement. Given a high-resolution color video and its corresponding low-quality depth video, we improve the quality of the depth video by increasing its resolution and suppressing noise. For that, a weighted mode filtering method is proposed based on a joint histogram. When the histogram is generated, the weight based on color similarity between reference and neighboring pixels on the color image is computed and then used for counting each bin on the joint histogram of the depth map. A final solution is determined by seeking a global mode on the histogram. We show that the proposed method provides the optimal solution with respect to L1 norm minimization. For temporally consistent estimate on depth video, we extend this method into temporally neighboring frames. Simple optical flow estimation and patch similarity measure are used for obtaining the high-quality depth video in an efficient manner. Experimental results show that the proposed method has outstanding performance and is very efficient, compared with existing methods. We also show that the temporally consistent enhancement of depth video addresses a flickering problem and improves the accuracy of depth video. [ABSTRACT FROM AUTHOR]

Liquid crystal display (LCD) devices are well known for their slow responses due to the physical limitations of liquid crystals. Therefore, fast moving objects in a scene are often perceived as blurred. This effect is known as the LCD motion blur. In order to reduce LCD motion blur, an accurate LCD model and an efficient deblurring algorithm are needed. However, existing LCD motion blur models are insufficient to reflect the limitation of human-eye-tracking system. Also, the spatiotemporal equivalence in LCD motion blur models has not been proven directly in the discrete 2-D spatial domain, although it is widely used. There are three main contributions of this paper: modeling, analysis, and algorithm. First, a comprehensive LCD motion blur model is presented, in which human-eye-tracking limits are taken into consideration. Second, a complete analysis of spatiotemporal equivalence is provided and verified using real video sequences. Third, an LCD motion blur reduction algorithm is proposed. The proposed algorithm solves an l_1-norm regularized least-squares minimization problem using a subgradient projection method. Numerical results show that the proposed algorithm gives higher peak SNR, lower temporal error, and lower spatial error than motion-compensated inverse filtering and Lucy–Richardson deconvolution algorithm, which are two state-of-the-art LCD deblurring algorithms. [ABSTRACT FROM PUBLISHER]