Search

International audience; When trying to independently apply image-trained algorithms to successive frames in videos, noxious flickering tends to appear. State-of-the-art post-processing techniques that aim at fostering temporal consistency, generate other temporal artifacts and visually alter the style of videos. We propose a postprocessing model, agnostic to the transformation applied to videos (eg style transfer, image manipulation using GANs, etc.), in the form of a recurrent neural network. Our model is trained using a Ping Pong procedure and its corresponding loss, recently introduced for GAN video generation, as well as a novel style preserving perceptual loss. The former improves long-term temporal consistency learning, while the latter fosters style preservation. We evaluate our model on the DAVIS and this http URL datasets and show that our approach offers state-of-the-art results concerning flicker removal, and better keeps the overall style of the videos than previous approaches.

Measuring video image quality is difficult and very often not precise. There are many factors that can affect the results and their interpretation. The advantages of subjective testing are in obtaining valid results for conventional and compressed television systems and the possibility of evaluating scalar MOS over a wide range of still and motion video image applications. Their disadvantages are in a wide variety of possible methods and tests to be considered, the high number of observers required and in high time demands. There are many objective testing approaches. It can be stated that the algorithms with video image feature analysis correlate better with subjective results than just simple pixel-based methods. A combination of different measurements and features gives the best results and correlation between subjective and objective scores but it is hardly technology independent.

This paper presents a study of a free convection flow around the walls of a ventilation mill of the Thermal Power Plant ?Kostolac B?, Kostolac, Serbia. A combined method consists of thermography and software post-processing, PATS. The PATS is specially developed for recognition of turbulence zones by the custom processing of large input data sets from thermal videos. The calculations determine maximum temperature fluctuation i.e. peak-to-peak fluctuation at every spot during the recording time. Three thermal videos of the walls were analyzed. Maximum temperature fluctuation occurred in the zones close to the obstacles, which are thus recognized as one of the main sources of turbulence. Besides, PATS has recognized fine camera oscillations and mechanical movements of a flexible material near the dozer wall. The detected zones of turbulence correspond to the previous studies and to the theory. The method shows good potential in the field of free convective flow research through the improvement of testing efficiency and cost savings. State-of-the-art thermograph cameras and updated software are recommended.

The illegal distribution of a digital movie is a common and significant threat to the film industry. With the advent of high-speed broadband Internet access, a pirated copy of a digital video can now be easily distributed to a global audience. A possible means of limiting this type of digital theft is digital video watermarking whereby additional information, called a watermark, is embedded in the host video. This watermark can be extracted at the decoder and used to determine whether the video content is watermarked. This paper presents a review of the digital video watermarking techniques in which their applications, challenges, and important properties are discussed, and categorizes them based on the domain in which they embed the watermark. It then provides an overview of a few emerging innovative solutions using watermarks. Protecting a 3D video by watermarking is an emerging area of research. The relevant 3D video watermarking techniques in the literature are classified based on the image-based representations of a 3D video in stereoscopic, depth-image-based rendering, and multi-view video watermarking. We discuss each technique, and then present a survey of the literature. Finally, we provide a summary of this paper and propose some future research directions.

This paper proposes a pseudo-dolly-in video generation method that reproduces motion parallax by applying image reconstruction processing to multi-view videos. Since dolly-in video is taken by moving a camera forward to reproduce motion parallax, we can present a sense of immersion. However, at a sporting event in a large-scale space, moving a camera is difficult. Our research generates dolly-in video from multi-view images captured by fixed cameras. By applying the Image-Based Modeling technique, dolly-in video can be generated. Unfortunately, the video quality is often damaged by the 3D estimation error. On the other hand, Bullet-Time realizes high-quality video observation. However, moving the virtual-viewpoint from the capturing positions is difficult. To solve these problems, we propose a method to generate a pseudo-dolly-in image by installing 3D estimation and image reconstruction techniques into Bullet-Time and show its effectiveness by applying it to multi-view videos captured at an actual soccer stadium. In the experiment, we compared the proposed method with digital zoom images and with the dolly-in video generated from the Image-Based Modeling and Rendering method., Published in: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) Date of Conference: 9-13 Oct. 2017 Conference Location: Nantes, France, 9780769563275

The depth-image-based rendering is a key technique for 3D video and free viewpoint video synthesis. One of the critical problems in current synthesis methods is that the background (BG) occluded by the foreground objects might be exposed in the new view, and some holes are produced in the synthesized video. However, most of the traditional hole-filling approaches may bring some blurry effect or artifacts in the virtual view. In this paper, a foreground removal approach for hole filling is proposed, in which the foreground objects are removed from both the 2D video and its corresponding depth map, and then a BG video and its depth map are generated before the 3D warping and used to eliminate the holes in the synthesized video. Moreover, a BG extension method is applied in the reference view to prevent the large holes occurring along the border areas in the virtual view. Our analysis and experimental results have indicated that the proposed approach has better performance compared with the other methods in terms of the quality of synthesized video, computational complexity, and running time in multiview synthesis or multiframe synthesis.

Recent works on video captioning mainly learn the map from low-level visual features to language description directly without explicitly representing the high-level semantic video concepts (e.g. objects, actions in the annotated sentences). To bridge the semantic gap, in this paper, addressing it, we propose a novel video attribute representation learning algorithm for video concept understanding and utilize the learned explicit video attribute representation to improve video captioning performance. To achieve it, firstly, inspired by the success of spectrogram in audio processing, a novel mid-level video representation named “video response map” (VRM) is proposed, by which the frame sequence could be represented by a single image representation. Therefore, the video attributes representation learning could be converted to a well-studied multi-label image classification problem. Then in the captions prediction step, the learned video attributes feature is integrated with the single frame feature to improve previous sequence-to-sequence language generation model by adjusting the LSTM (Long-Short Term Memory) input units. The proposed video captioning framework could both handle variable frame inputs and utilize high-level semantic video attribute features. Experimental results on video captioning tasks show that the proposed method, utilizing only RGB frames as input without extra video or text training data, could achieve competitive performance with state-of-the-art methods. Furthermore, the extensive experimental evaluations on the UCF-101 action classification benchmark well demonstrate the representation capability of the proposed VRM.

A new image enhancement algorithm employing guided filtering is proposed in this work for enhancement of solar images and videos, so that users can easily figure out important fine structures imbedded in the recorded images/movies for solar observation. The proposed algorithm can efficiently remove image noises, including Gaussian and impulse noises. Meanwhile, it can further highlight fibrous structures on/beyond the solar disk. These fibrous structures can clearly demonstrate the progress of solar flare, prominence coronal mass emission, magnetic field, and so on. The experimental results prove that the proposed algorithm gives significant enhancement of visual quality of solar images beyond original input and several classical image en-hancement algorithms, thus facilitating easier determi-nation of interesting solar burst activities from recorded images/movies.

View synthesis prediction (VSP) is a crucial coding tool for improving compression efficiency in the next generation 3D video systems. However, VSP is susceptible to catastrophic error propagation when multi-view video plus depth (MVD) data are transmitted over lossy networks. This paper aims at accurately modeling the transmission errors propagated in the inter-view direction caused by VSP. Toward this end, we first study how channel errors gradually propagate along the VSP-based inter-view prediction path. Then, a new recursive model is formulated to estimate the expected end-to-end distortion caused by those channel losses. For the proposed model, the compound impact of the transmission distortions of both the texture video and depth map on the quality of the synthetic reference view is mathematically analyzed. Especially, the expected view synthesis distortion due to depth errors is characterized in the frequency domain using a new approach, which combines the energy densities of the reconstructed texture image and the channel errors. The proposed model also explicitly considers the disparity rounding operation invoked for the sub-pixel precision rendering of the synthesized reference view. Experimental results are presented to demonstrate that the proposed analytic model is capable of effectively modeling the channel-induced distortion for MVD-based 3D video transmission.

Soft errors resulting from encoding video sequences on unreliable hardware can create significant artifacts in decoded video sequences, contributing to extreme video quality degradation. Modern systems are required to operate under increasingly challenging constraints, including smaller feature sizes and lower operating voltage, increasing the likelihood of soft errors in the video encoding hardware. These conditions are of particular concern for energy-limited battery-operated systems since they may be required to operate in nonideal environments and/or continue operating with a practically depleted energy source. The proposed parallel independent signature processing design performs error detection and mitigation in video encoding hardware, enabling a graceful degradation of quality when encoding by using unreliable hardware. The effects of soft errors are minimized by preventing the error propagation normally associated with errors in encoded video sequences. This allows for the recovery of quality when errors are present in the video encoding system. Conventional video encoding techniques are designed to handle worst case error rates by increasing gate sizes and/or increasing the operating voltage of the system. Such designs have error-rate limits, and when these limits are reached, the systems tend to fail catastrophically, resulting in an unrecoverable signal. The proposed design allows for single upset events to translate into single transient artifacts in a decoded video sequence.

A new algorithm and its hardware architecture are presented to up-scale high-definition (HD) and full-HD video streams to 4-K ultra-HD video streams in real time. The Lagrange interpolation is employed, as it provides high estimation accuracy and hardware-friendly properties. To enhance the accuracy further, the pixels at the edge regions are specially processed by employing an image-sharpening technique. Experimental results show that the proposed architecture provides the best visual quality at the cost of reasonable hardware resources.

Offline or deferred solutions are frequently employed for high quality and reliable results in current video stabilization. However, neither of these solutions can be used for strict real-time applications. In this paper, we propose a practical and robust algorithm for real-time video stabilization. To achieve this, a novel and efficient motion model based on inter-frame homography estimation is proposed to represent the video motion. An important feature of the proposed motion model is that it updates at each frame input to reduce the accumulation errors caused by parallax or scene changes. We also propose a novel Kalman filter for the motion smoothing and a unique mosaic algorithm for the video completion. The proposed Kalman filter and mosaic algorithm enable the development of a practical real-time video stabilizer that not only produces steady video but also retains the full resolution of the original video. We verify the proposed algorithm through a broad range of video sequences that demonstrate that the proposed algorithm is computationally efficient while being able to robustly stabilize videos with various challenges.

We present a video vectorization method that generates a video in vector representation from an input video in raster representation. A vector-based video representation offers the benefits of vector graphics, such as compactness and scalability. The vector video we generate is represented by a simplified tetrahedral control mesh over the spatial-temporal video volume, with color attributes defined at the mesh vertices. We present novel techniques for simplification and subdivision of a tetrahedral mesh to achieve high simplification ratio while preserving features and ensuring color fidelity. From an input raster video, our method is capable of generating a compact video in vector representation that allows a faithful reconstruction with low reconstruction errors.

Here we present results of research into multi-agent approach to solving the problem of constructing a wireless Ad Hoc network to transmit video stream during optical navigation. When obtaining data from video cameras in real time, there is a problem of realization of effective coordination of video data. To improve the algorithms of capturing an object, it is necessary to improve the architecture of devices that utilize network metadata for the analysis of scenes from video surveillance. As a result of exploring the ways of solving the specified problems, we proposed a method for active coordination of video cameras based on the multi-agent approach (MAC-on-MAA). The application of this method increases efficiency of network cameras through their reconfiguration. We examined the problem of determining an adequate level of illuminance normalization of a video frame based on analysis of the modules throughput of multi-agent system for encoding transmitted video data. It was revealed that the time of analysis of video stream from a network decreases when ignoring the decoding of certain domains of graphic elements in the received video frames. The devised MAC-on-MAA method allows effective updating of routing tables in the reactive protocol AODV. We obtained analytic expressions for the assessment of quantitative characteristics of energy consumption by mobile agents. Protection of communication channels of video surveillance network is improved through the integration of steganocontainers in the free bits of RGB characteristics of video frames pixels.

Full-reference (FR) video quality estimators (QEs) resize either the distorted input video or the reference video to compute the quality when these videos have different spatial resolutions. This resizing operation causes several limitations. Multiscale Image Quality Estimator (MIQE) overcomes those limitations for images, but it does not consider the temporal characteristics of video. In this paper, we develop an FR video QE that integrates MIQE with the motion information to estimate the quality of the distorted video without resampling the reference or the test videos. We also perform subjective tests to compare the proposed algorithm with the existing QEs. In these tests, the reference and the input videos are displayed at their native resolutions. The test results show that the proposed algorithm outperforms other QEs when the reference video and the input video have different spatial resolutions. We have also evaluated the performance of the approach using the Scalable Video Database.

With the advance of coding and network technology, the multiview video (MVV) system has received growing attention in the 3-D TV market since it can provide consumers with more realistic scenes from different viewpoints. In the MVV system, each video sequence consists of one color-map video and one depth video to be encoded and the depth-image-based rendering technique is often used to create/synthesize virtual views for different viewpoints. To attain the same quality of virtual views, encoding depth videos in a fast and bitrate-saving manner is crucial. Based on the depth no-synthesis-error model developed by Zhao et al. , this paper proposes a new intra-/inter-prediction scheme first and then a fast quadtree structure determination scheme for encoding depth videos in 3-D High Efficiency Video Coding (3D-HEVC). The first proposed scheme has bitrate-saving merit because for each coding unit, each depth value is modified to approach the predicted intra-/inter-depth value as closely as possible, but without causing any synthesis errors in the resultant virtual views. The second proposed scheme has the merit of low computational cost because it can terminate the quadtree structure determination for coding tree units as early as possible. Based on four typical test video sequences, the empirical results demonstrate that on average, our proposed encoding method for depth videos has 16.3% bitrate and 13.8% encoding-time improvement ratios when compared with the traditional method in 3D-HEVC test model, while preserving the quality of the rendered virtual views. In addition, our proposed method outperforms that of Lee et al. in terms of the required bitrate and encoding time.

Due to the popularity of online video sharing websites such as YouTube, millions of users have treated online video as a source of information and entertainment. Therefore, video annotation has evoked great interest in the past few years. In this paper, we propose a four-step approach to automatically annotate video shots with sentences. The first step is video preprocessing, converting video shot into a sequence of frame images. The second step is to find related candidate elements of the sentence about the video contents. The main elements in the sentence are objects, events, scenes, and modifiers. These candidate elements are gained by searching for similar images with the video frames in our collected image data sets instead of video data sets. The third step is to select the best elements among these candidate ones by a weighted scoring algorithm. The final step is to construct a sentence with the help of a correlation graph algorithm to analyze the relationships among the best elements. The experimental results indicate that our method is effective to annotate videos with sentences. What is more, the weighted scoring algorithm and the correlation graph algorithm that we propose are efficient in developing the experimental performance.

Rate control in video compression adjusts the encoding parameters to reach a certain target bitrate for the encoded video. State-of-the-art rate controllers for hybrid video coding typically employ content-dependent video bitrate models and video quality metrics (VQMs). To capture the content characteristics, temporal and spatial video activity measures are determined from the raw video using computationally complex algorithms that require access to the uncompressed source video. In automotive deployments, however, full access to the uncompressed source video and the internal functions of video encoders is typically not possible. As a remedy, in this paper, we present a low-complexity approach to estimate the temporal activity (TA) and spatial activity (SA) measures for videos that are captured by a front-facing camera of a vehicle, based on the context information of the vehicle. To this end, we exploit information about the dynamics of the vehicle and other vehicles in the field-of-view of the front-facing camera. We apply the estimated TA and SA values to a video bitrate model and an objective VQM and use these models to solve the rate control problem to determine the optimal encoding settings for given bitrate constraints. The proposed low-complexity solution offers a similar accuracy in achieving rate constraints and similar perceptual quality characteristics as a solution that uses the computed TA and SA values, with the advantage that no access to the uncompressed source video stream or the internal functions of the video encoder is required.

To manipulate large video contents, effective video indexing and retrieval are required. A large number of video indexing and retrieval algorithms have been presented for frame-wise user query or video content query whereas a relatively few video sequence matching algorithms have been proposed for video sequence query. In this paper, we propose an efficient algorithm that extracts key frames using color histograms and matches the video sequences using edge features. To effectively match video sequences with a low computational load, we make use of the key frames extracted by the cumulative measure and the distance between key frames, and compare two sets of key frames using the modified Hausdorff distance. Experimental results with real sequence show that the proposed video sequence matching algorithm using edge features yields the higher accuracy and performance than conventional methods such as histogram difference, Euclidean metric, Battachaya distance, and directed divergence methods.

This article introduces interactive video maps for the web. The main component of video maps is a video stream that is areally georeferenced to a spatial reference system in the same way rectified raster orthoimages are georeferenced. The areal georeference allows for interactivity that goes beyond the play, pause, and stop functionality of video player software. We highlight two types of functionality, allowing the user to (1) combine the video stream layer with other raster and vector map layers and (2) adjust the projection of the map in real time. We exemplify video maps by A Year in the Life of Earth's CO2, an interactive video map that visualizes the results of a high-resolution NASA computer model of global atmospheric carbon dioxide distribution. The map shows how carbon dioxide travels around the globe over the course of one year. We use a combination of WebGL, a programming interface to the hardware-accelerated graphics pipeline, and HTML5 video for adding an areally georeferenced video ...

This paper investigates a novel learning-based camera motion characterization scheme along with its application to the video stabilization problem. The proposed characterization scheme represents the compressed domain block motion vectors (MVs) using polar angle and magnitude histograms. Discriminative features from these two histograms are extracted and fed to a supervised learning-based hierarchical classifier for recognizing the six camera motion patterns. A comparative analysis with an existing scheme is carried out to support and validate the proposed characterization scheme. The proposed scheme works at the frame level by classifying the inter-frame camera motion patterns. This scheme is extended to classify the video segments and a novel application to video stabilization is investigated. An experimental analysis of a number of test sequences captured using a handheld video camera shows that by characterizing the smooth and jittery motions, selective video stabilization could be carried out only on those video segments that have been degraded. This approach of selective video stabilization saves considerable amount of computational time compared with running the stabilization algorithm on the entire video sequence, as proposed in the literature. The proposed strategy of using a classification scheme prior to applying the video stabilization routine offers a new paradigm to the conventional video stabilization problem. Experimental validation carried out using exhaustive search motion estimation obtained block MVs, and H.264/Advanced Video Coding-obtained MVs shows that by using the idea of selective video stabilization, up to 62% reduction in processing time can be achieved compared with video stabilization approaches wherein the entire video sequence is processed.

A comprehensive scheme of temporal profile of surveillance video is proposed.It provides the sampling selection, analyzes properties of shape and motion of the temporal profile, develop the algorithms to detect dynamic targets, and measure their motion direction.Finally, the visualization of targets with multiple temporal slices with transparency is provided.Further, robust object detection based on background updating, and real time data transmission of is given.A comparison of our method with other video indexing methods is also given. This work converts the surveillance video to a temporal domain image called temporal profile that is scrollable and scalable for quick searching of long surveillance video by human operators. Such a profile is sampled with linear pixel lines located at critical locations in the video frames. It has precise time stamp on the target passing events through those locations in the field of view, shows target shapes for identification, and facilitates the target search in long videos. In this paper, we first study the projection and shape properties of dynamic scenes in the temporal profile so as to set sampling lines. Then, we design methods to capture target motion and preserve target shapes for target recognition in the temporal profile. It also provides the uniformed resolution of large crowds passing through so that it is powerful in target counting and flow measuring. We also align multiple sampling lines to visualize the spatial information missed in a single line temporal profile. Finally, we achieve real time adaptive background removal and robust target extraction to ensure long-term surveillance. Compared to the original video or the shortened video, this temporal profile reduced data by one dimension while keeping the majority of information for further video investigation. As an intermediate indexing image, the profile image can be transmitted via network much faster than video for online video searching task by multiple operators. Because the temporal profile can abstract passing targets with efficient computation, an even more compact digest of the surveillance video can be created.

Reconstructing a full-resolution stereoscopic video from an asymmetric stereoscopic video is a challenging task. The existing approaches require depth information, which imposes an additional challenge in data acquisition. In this paper, we propose a novel scheme that is capable of obtaining super-resolution and depth estimation simultaneously from an asymmetric stereoscopic video. The proposed scheme models the video super-resolution and stereo matching with a unified energy function. Then, we apply an alternating optimization method to minimize this energy function, which can be implemented with a two-step algorithm. In the first step we calculate the initial depth map by using a region-based cooperative optimization technique while considering the temporal consistency in video. In the second step we resolve the super-resolution problem under the guidance of the depth information. It is effective because each step benefits from the additional improvement over the previous step. We iteratively update the two steps until stable depth and super-resolution results are obtained. We have conducted a series of experiments on public stereoscopic video sequences to evaluate the performance of the proposed method. Both objective indexes and subjective visual comparisons verify that the proposed scheme can achieve satisfactory super-resolution results and high-quality depth map simultaneously. In particular, the subjective evaluation experiments on a 3-D monitor show that this scheme outperforms others and achieves the best visual sharpness.

A video is an effective tool to exchange the information in the structure of showing the brief text message due to the advance developed technology. Video capturing is effortless process but the related video retrieval is the difficult process, for that process the videos must be indexed. Retrieval is the method that retrieved a video using a user query. The query will be image or texts depend upon the query result output system that returned a particular video or image based on that query. In this project we create a indexing for video file by using segment based indexing technique. Here video will be divided into a hierarchy which is in storyboards of film making. For instance, a hierarchical based video search is composed into multi stage abstraction for assist the users to locate the specific video segments/frames logically. This paper brings out the reduced bandwidth and reduced delays the video through the network of searching and reviewing. Experimental results verify this.

This paper proposes a novel representation method which is called Depth and residual images based representation (DRIBR) for virtual image rendering in 3D video. The proposed DRIBR is composed of three parts: a single view image from one particular viewpoint, its corresponding depth map as well as a residual image. The residual image is pre-calculated from the combination of the above two components plus an image taken from the other viewpoint. The proposed DRIBR achieves low data redundancy, and resolves the occlusion problem which is one of the most annoying artifacts during the rendering of virtual image in traditional Depth image based rendering (DIBR). This paper addresses details of DRIBR and virtual image rendering algorithms based on the proposed format. We test the DRIBR on various stereo datasets and the experimental results demonstrate that our method can achieve better performance on both subjective and objective quality assessment.

Cloud gaming allows games to be rendered on the cloud server and allows the rendered videos to be encoded and streamed in real time to the player’s devices. Compared with other video streaming applications, cloud gaming offers a unique opportunity to enhance the video encoding process by exploiting rendering information. In this paper, we propose two techniques to improve cloud gaming video encoding, aiming at enhancing the perceived video quality and reducing the computational complexity, respectively. First, we develop a rendering-based prioritized encoding technique to improve the perceived game video quality according to network bandwidth constraints. We first propose a technique to generate a macroblock (MB)-level saliency map for every game video frame using rendering information. Furthermore, based on such a saliency map, a prioritized rate allocation scheme is proposed to dynamically adjust the value of quantization parameter of each MB. The experimental results indicate that the perceptual quality can be greatly improved using the proposed technique. We also develop a rendering-based encoding acceleration technique that utilizes rendering information to reduce the computational complexity of video encoding. This technique mainly consists of two parts. First, we propose a method to directly calculate the motion vectors (MVs) without employing the compute intensive motion search procedure. Second, based on the computed MVs, we propose a fast mode selection algorithm to reduce the number of candidate modes of each MB. The experimental results show that the proposed technique can achieve more than 42% saving in encoding time with very limited degradation in video quality.

The current research concerns the problem of video stabilization “in a point”, which aims to stabilize all video frames according to one chosen reference frame to produce a new video, as by a static camera. Similar task importance relates providing static background in the video sequence that can be usable for correct measurements in the frames when studying dynamic objects in the video. For this aim we propose an efficient combined approach, called “3×3OF9×9”. It fuses our the previous development for fast and rigid 2D video stabilization [2] with the well-known Optical Flow approach, applied by parts via Otsu segmentation, for eliminating the influence of moving objects in the video. The obtained results are compared with those, produced by the commercial software Warp Stabilizer of Adobe-After-Effects CS6.

The Visual Computer, 22 (9-11), ISSN:0178-2789, ISSN:1432-2315

Currently, there is a rapid development in the techniques of the automated image based modelling (IBM), especially in advanced structure-from-motion (SFM) and dense image matching methods, and camera technology. One possibility is to use video imaging to create 3D reality based models of cultural heritage architectures and monuments. Practically, video imaging is much easier to apply when compared to still image shooting in IBM techniques because the latter needs a thorough planning and proficiency. However, one is faced with mainly three problems when video image sequences are used for highly detailed modelling and dimensional survey of cultural heritage objects. These problems are: the low resolution of video images, the need to process a large number of short baseline video images and blur effects due to camera shake on a significant number of images. In this research, the feasibility of using video images for efficient 3D modelling is investigated. A method is developed to find the minimal significant number of video images in terms of object coverage and blur effect. This reduction in video images is convenient to decrease the processing time and to create a reliable textured 3D model compared with models produced by still imaging. Two experiments for modelling a building and a monument are tested using a video image resolution of 1920×1080 pixels. Internal and external validations of the produced models are applied to find out the final predicted accuracy and the model level of details. Related to the object complexity and video imaging resolution, the tests show an achievable average accuracy between 1 – 5 cm when using video imaging, which is suitable for visualization, virtual museums and low detailed documentation.

Nowadays, broadcasters deliver ultra-high resolution video to their consumers. This live video is sent to a set-top box for display on a television. However, if one or more users in the home want to view the same video on their personal mobile devices with a lower display resolution and limited processing power, decoding the original ultra-high resolution video would result in stuttering and quickly drain the battery life on these devices. To enable a satisfactory consumer experience, the resolution of the video stream should be adapted to the target mobile device at the set-top box. The aim of this paper is to investigate the performance of different machine learning strategies to arbitrary downsize video pre-encoded with the high efficiency video coding standard (HEVC). These machine learning techniques exploit correlation between input and output coding information to predict the splitting behavior of HEVC coding units. Several machine learning algorithms are optimized. Additionally, both online and offline training strategies are tested. Of the tested algorithms, online-trained random forests achieve the best compression-efficiency with a bit rate increase of 5.4% and an average complexity reduction of 70%1.

The data is exploding day by day in digital technology. Now a day’s multimedia data is also handled by the database, multimedia data contains data like images, text and video. The video processing plays a tremendous role in the multimedia but all the videos are not same, it can exists number of settings and different number of formats. By This video processing system the video is processed for enhancement, analysis, dividing the channels and binarization by using different image processing techniques. In this system different color system like YCBR, HSL, and RGB color systems are considered for processing any type of video. For this system, the input video can be from a stored file or continuous stream of video sequences from the web camera (or) any type of camera by this video processing system we can improve the quality of the video and we can also apply some special effects to the video by applying various image processing techniques and filters. The enhancement techniques considered in his system are filtering with correlation and convolution, adaptive smoothing, conservative smoothing and median filtering. The analysis techniques like edge detection, histogram and statistical analysis are considered for this system. Binarization methods implemented in this system are Custom Threshold, Order Dither. The Color filters like converting RGB to Grayscale, Grayscale to RGB ,Sepia, invert, rotate, Custom Color filter, Euclidean color filter, channel filter, red, green, blue, cyan, magenta and yellow, they are so many other filters are also implemented in this system.

HEVC and VP9 are the current state-of-the-art in video compression, since their bit-streams were recently finalized in January and May 2013, respectively. These codecs are the generational successors of the currently most widely-used video codecs, H.264/AVC and VP8, respectively. Consequently, it is expected that in the near future these new codecs will replace their predecessors. However, the process of converting video contents compressed with one standard to those using another standard is highly computationally expensive, since a priori the video contents must be decompressed and compressed with the target video encoder. Nevertheless, it is known that some information can be extracted from the decoding process in order to accelerate the encoding process. In this paper, some techniques for transcoding from HEVC to VP9 are presented. By using some information extracted from the HEVC decoding process some coding modes will be discarded from being checked in the VP9 encoder. By applying the proposed approaches, a reduction of about 60% of the coding complexity is achieved with acceptable Rate Distortion penalties.

The latest High Efficiency Video Coding (HEVC) standard achieves a significant compression efficiency improvement over the H.264/Advanced Video Coding (AVC) standard, but with a much higher computational complexity. In this paper, we propose a novel framework for software-based H.264/AVC to HEVC transcoding, integrated with tools such as wavefront parallel processing that are useful for achieving higher levels of parallelism on multicore processors and distributed systems. By utilizing information extracted from the input H.264/AVC bitstream, the transcoding process can be greatly accelerated with a visual quality loss that is modest for many applications. Based on the HEVC HM 14.0 reference software and using standard HEVC test bitstreams, the proposed transcoder can achieve up to $60\times $ speedup on a Quad Core 8-thread server over decoding–re-encoding based on FFMPEG and the HM software with a BD-rate loss of 15%–20%. By implementing a group of picture-level task distribution on a distributed system with nine processing units, the proposed software transcoder can achieve a speed for transcoding 720 p at 30 Hz in real time.

Many compelling video processing effects can be achieved if per-pixel depth information and 3D camera calibrations are known. However, the success of such methods is highly dependent on the accuracy of this "scene-space" information. We present a novel, sampling-based framework for processing video that enables high-quality scene-space video effects in the presence of inevitable errors in depth and camera pose estimation. Instead of trying to improve the explicit 3D scene representation, the key idea of our method is to exploit the high redundancy of approximate scene information that arises due to most scene points being visible multiple times across many frames of video. Based on this observation, we propose a novel pixel gathering and filtering approach. The gathering step is general and collects pixel samples in scene-space, while the filtering step is application-specific and computes a desired output video from the gathered sample sets. Our approach is easily parallelizable and has been implemented on GPU, allowing us to take full advantage of large volumes of video data and facilitating practical runtimes on HD video using a standard desktop computer. Our generic scene-space formulation is able to comprehensively describe a multitude of video processing applications such as denoising, deblurring, super resolution, object removal, computational shutter functions, and other scene-space camera effects. We present results for various casually captured, hand-held, moving, compressed, monocular videos depicting challenging scenes recorded in uncontrolled environments.

The prevalence of video recording capability, either on surveillance systems or mobile devices, has contributed to the popularity of video data. As a result, video management has become relatively more important than before and, in particular, video retrieval has been one of the main issues in this regard. Traditional video retrieval systems take texts as the inputs to look for similar information from the title, annotation or embedded textual data of a video, in a way that is very similar to the keyword search adopted by a common search engine. However, the lack of visual information specification during a search often makes the result rather inaccurate or even useless. For this reason, video retrieval systems using images or videos as the inputs have also been proposed; nevertheless, the associated ambiguity and complexity have made the implementation of such systems relatively difficult and, therefore, those systems are not as successful as desired. To address this, in this paper, we propose to perform a video retrieval of a desired object through the inputs of its trajectory and/or appearance , together with the help of a 3-D graphical user interface for more intuitive interactions, so that more satisfactory results can be achieved. We firmly believe that such a framework could serve as the foundation for behavior analysis used in many surveillance systems.

Recent advancement in cameras and image processing technology has generated a paradigm shift from traditional 2D and 3D video to multi-view video (MVV) technology, while at the same time improving video quality and compression through standards such as high efficiency video coding (HEVC). In multi-view, cameras are placed in predetermined positions to capture the video from various views. Delivering such views with high quality over the Internet is a challenging prospect, as MVV traffic is several times larger than traditional video, since it consists of multiple video sequences, each captured from a different angle, requiring more bandwidth than single-view video to transmit MVV. In addition, the Internet is known to be prone to packet loss, delay, and bandwidth variation, which adversely affect MVV transmission. Another challenge is that end users' devices have different capabilities in terms of computing power, display, and access link capacity, requiring MVV to be adapted to each user's context. In this paper, we propose an HEVC multi-view system using Dynamic Adaptive Streaming over HTTP to overcome the above-mentioned challenges. Our system uses an adaptive mechanism to adjust the video bit rate to the variations of bandwidth in best effort networks. We also propose a novel scalable way for the multi-view video and depth content for 3D video in terms of the number of transmitted views. Our objective measurements show that our method of transmitting MVV content can maximize the perceptual quality of virtual views after the rendering and hence increase the user's quality of experience.

With the arrival of auto-stereoscopic 3D displays for mobile devices, and emergence of more 3D content, there is much anticipation for 3D mobile multimedia experiences, including 3D display gaming. Simultaneously, with the emergence of cloud computing, more mobile applications are being developed to take advantage of the elastic cloud resources. In this paper, we explore the possibility of developing Cloud Mobile 3D Display Gaming, where the 3D video rendering and encoding is performed on cloud servers, with the resulting 3D video streamed over wireless networks to mobile devices with 3D displays for a true 3D mobile gaming experience. However, with the significantly higher bitrate requirement for 3D video, ensuring user experience may be a challenge, both in terms of 3D video quality and network delay (response time), considering the bandwidth constraints and fluctuations of wireless networks. In this paper, we propose a new asymmetric graphics rendering approach which can significantly reduce the video encoding bitrate needed for a certain video quality, thereby making it easier to transmit the video over wireless network. However, since asymmetric graphics rendering may also impair the graphics quality, we need to be able to understand and measure its impact. We conduct subjective tests to study and model the impairments due to asymmetric graphics rendering and network delay, thereby developing a user experience model for cloud based mobile 3D display gaming. By conducting subsequent subjective tests, we prove the correctness of the impairment functions and the resulting user experience model. Furthermore, given any network condition, we propose to solve the problem of selecting the optimal graphics rendering factors for the left and right views so as to maximize user experience of cloud mobile 3D display gaming. In order to solve this problem, we first develop a model to estimate the resulting video bitrate of the rendered 3D video when certain graphics rendering factors are used. Next, we derive a model to predict the delay given the available network bandwidth and the video bitrate of the rendered 3D video. We use the above two models together with a branch and bound algorithm to solve the optimization problem and determine the optimal values for the left and right view rendering factors. Experiments conducted using real 4G-LTE network profiles on commercial cloud service demonstrate the feasibility of significant improvement in user experience when the proposed optimization algorithm is used to dynamically select optimal rendering factors according to changing network conditions.

We present a novel reconfigurable hardware architecture for interpolation filtering in high efficient video coding that adapts to run-time changes of the number of interpolation filter calls and thereby provides a high potential of energy efficiency. It employs a picture-based prediction scheme to estimate the number of interpolation filter calls at run-time by monitoring the group of pictures history based on video coding structure knowledge. Reconfigurable acceleration engines are developed that can adapt to different filter types. Dynamic composition of different instances of these engines enables different implementation versions with area versus throughput tradeoff. A run-time selection scheme determines the best implementation version for each picture based on the throughput requirements. Compared to state-of-the-art, our architecture reduces resource usage by 57% while supporting various throughputs and video resolutions.

In practical multimedia systems, the content of coded video streams often needs to be re-edited at the nodes of transmitting networks. For example, logo insertion is always required for copyright protection at different local transmitting nodes. This kind of video stream editing is denoted as video content editing transcoding (VCET) in this paper. Though some techniques have been suggested for VCET, these methods cannot meet the requirement of dynamic transmitting bandwidth in practical applications. In this paper, we proposed an interface macroblock-based transcoding scheme for VCET, which can reuse the variable length codes of the original video streams as much as possible to achieve the best VCET quality. In order to ensure that the edited video streams can be transmitted by the original bandwidth, we also proposed a rate control algorithm for VCET, which can accurately control the bitrate of edited video streams according to the frame level coding bits of the original video streams. Experimental results showed that the proposed scheme achieved substantially better results in bitrate accuracy, computational complexities, and video quality than many other existing schemes.

This paper proposes a method for identifying the video that retains the most information from its parent video. Since the parent video is often unavailable, the proposed method estimates its content through the collaborative use of the available video signals that are edited copies of the parent video. By reducing the difference between the video signals of the edited videos, the proposed method then enables the use of conventional no-reference video quality assessment algorithms. Since editing a video requires recompressing it, and since quality assessment algorithms can detect signs of recompression, the proposed method can identify the edited video that retains the most information from the parent video. The effectiveness of the proposed method is verified by subjective experiments over artifical and real-world data sets that include a total of over 400 videos.

Bit rate is one of the important criterions for digital video quality. With some video tools, however, video bit rate can be easily increased without improving the video quality at all. In such a case, a claimed high bit rate video would actually have poor visual quality if it is up-converted from an original lower bit rate version. Therefore, exposing fake bit rate videos becomes an important issue for digital video forensics. To the best of our knowledge, although some methods have been proposed for exposing fake bit rate MPEG-2 videos, no relative work has been reported to further estimate their original bit rates. In this paper, we first analyze the statistical artifacts of these fake bit rate videos, including the requantization artifacts based on the first-digit law in the DCT frequency domain (12-D) and the changes of the structural similarity indexes between the query video and its sequential bit rate down-converted versions in the spatial domain (4-D), and then we propose a compact yet very effective 16-D feature vector for exposing fake bit rate videos and further estimating their original bit rates. The extensive experiments evaluated on hundreds of video sequences with four different resolutions and two typical compression schemes (i.e., MPEG-2 and H.264/AVC) have shown the effectiveness of the proposed method compared with the existing relative ones.

For a real-time video application such as a remote learning session, the main difficulty is to keep the latency low while providing a video stream to viewers with different devices over a variety of bandwidth restricted networks. This Letter proposes an optimisation to dynamically select a subset of predefined different video representations, such that with a limited number of encoders, the largest possible group of viewers receives a decodable video stream at a low latency. This optimisation outperforms the static encoder approach by achieving a fairness of 98.2% with an average client bandwidth utilisation of 84.1% to provide higher quality video.

Video frame interpolation typically involves two steps: motion estimation and pixel synthesis. Such a two-step approach heavily depends on the quality of motion estimation. This paper presents a robust video frame interpolation method that combines these two steps into a single process. Specifically, our method considers pixel synthesis for the interpolated frame as local convolution over two input frames. The convolution kernel captures both the local motion between the input frames and the coefficients for pixel synthesis. Our method employs a deep fully convolutional neural network to estimate a spatially-adaptive convolution kernel for each pixel. This deep neural network can be directly trained end to end using widely available video data without any difficult-to-obtain ground-truth data like optical flow. Our experiments show that the formulation of video interpolation as a single convolution process allows our method to gracefully handle challenges like occlusion, blur, and abrupt brightness change and enables high-quality video frame interpolation., CVPR 2017, http://graphics.cs.pdx.edu/project/adaconv

We propose a technique that propagates information forward through video data. The method is conceptually simple and can be applied to tasks that require the propagation of structured information, such as semantic labels, based on video content. We propose a 'Video Propagation Network' that processes video frames in an adaptive manner. The model is applied online: it propagates information forward without the need to access future frames. In particular we combine two components, a temporal bilateral network for dense and video adaptive filtering, followed by a spatial network to refine features and increased flexibility. We present experiments on video object segmentation and semantic video segmentation and show increased performance comparing to the best previous task-specific methods, while having favorable runtime. Additionally we demonstrate our approach on an example regression task of color propagation in a grayscale video., Comment: Appearing in Computer Vision and Pattern Recognition, 2017 (CVPR'17)