Search

International audience; When trying to independently apply image-trained algorithms to successive frames in videos, noxious flickering tends to appear. State-of-the-art post-processing techniques that aim at fostering temporal consistency, generate other temporal artifacts and visually alter the style of videos. We propose a postprocessing model, agnostic to the transformation applied to videos (eg style transfer, image manipulation using GANs, etc.), in the form of a recurrent neural network. Our model is trained using a Ping Pong procedure and its corresponding loss, recently introduced for GAN video generation, as well as a novel style preserving perceptual loss. The former improves long-term temporal consistency learning, while the latter fosters style preservation. We evaluate our model on the DAVIS and this http URL datasets and show that our approach offers state-of-the-art results concerning flicker removal, and better keeps the overall style of the videos than previous approaches.

This paper proposes a pseudo-dolly-in video generation method that reproduces motion parallax by applying image reconstruction processing to multi-view videos. Since dolly-in video is taken by moving a camera forward to reproduce motion parallax, we can present a sense of immersion. However, at a sporting event in a large-scale space, moving a camera is difficult. Our research generates dolly-in video from multi-view images captured by fixed cameras. By applying the Image-Based Modeling technique, dolly-in video can be generated. Unfortunately, the video quality is often damaged by the 3D estimation error. On the other hand, Bullet-Time realizes high-quality video observation. However, moving the virtual-viewpoint from the capturing positions is difficult. To solve these problems, we propose a method to generate a pseudo-dolly-in image by installing 3D estimation and image reconstruction techniques into Bullet-Time and show its effectiveness by applying it to multi-view videos captured at an actual soccer stadium. In the experiment, we compared the proposed method with digital zoom images and with the dolly-in video generated from the Image-Based Modeling and Rendering method., Published in: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) Date of Conference: 9-13 Oct. 2017 Conference Location: Nantes, France, 9780769563275

Device screen size, design layout and resolution are some of the device context parameters which impact video quality on mobile devices. In this paper, through subjective tests, we evaluate the impact of mobile device pixel density on the video quality. We then, through the subjective test, propose a video quality model that takes pixel density into consideration. The proposed model was evaluated, and the preliminary results have shown that the proposed model performs well regarding correlation coefficient. The model has potential use for multimedia service providers such as YouTube and Netflix in areas of quality control and optimization of network and multimedia services.

Video frame interpolation typically involves two steps: motion estimation and pixel synthesis. Such a two-step approach heavily depends on the quality of motion estimation. This paper presents a robust video frame interpolation method that combines these two steps into a single process. Specifically, our method considers pixel synthesis for the interpolated frame as local convolution over two input frames. The convolution kernel captures both the local motion between the input frames and the coefficients for pixel synthesis. Our method employs a deep fully convolutional neural network to estimate a spatially-adaptive convolution kernel for each pixel. This deep neural network can be directly trained end to end using widely available video data without any difficult-to-obtain ground-truth data like optical flow. Our experiments show that the formulation of video interpolation as a single convolution process allows our method to gracefully handle challenges like occlusion, blur, and abrupt brightness change and enables high-quality video frame interpolation., CVPR 2017, http://graphics.cs.pdx.edu/project/adaconv

We propose a technique that propagates information forward through video data. The method is conceptually simple and can be applied to tasks that require the propagation of structured information, such as semantic labels, based on video content. We propose a 'Video Propagation Network' that processes video frames in an adaptive manner. The model is applied online: it propagates information forward without the need to access future frames. In particular we combine two components, a temporal bilateral network for dense and video adaptive filtering, followed by a spatial network to refine features and increased flexibility. We present experiments on video object segmentation and semantic video segmentation and show increased performance comparing to the best previous task-specific methods, while having favorable runtime. Additionally we demonstrate our approach on an example regression task of color propagation in a grayscale video., Comment: Appearing in Computer Vision and Pattern Recognition, 2017 (CVPR'17)

A video segmentation method based on strong target constrained video saliency (STCVS) is proposed in this paper. In order to detect the salient region fast and effectively, the proposed STCVS is extracted based on the extension of image saliency by enforcing the salient region constrained with the location, scale and color model of the target. Besides, according to the results of STCVS, the super-pixel based spatial-temporal DenseCut method is given for video segmentation, where the super-pixel is regarded as basic unit instead of pixel due to the large amount of data in video, and the pairwise term of inter-frame is included for ensuring temporal and spatial continuity of video segmentation results. Compared with some state-of-art video segmentation methods on DAVIS dataset, our proposed method performs outstandingly on accuracy, efficiency and robustness.

As more and more coding standards are developed for videos with higher resolution and higher frame rates, the conversion between them is in demand. This paper focuses on the development of fast VP9-to-HEVC transcoding algorithm. Since the decoding-and-full-re-encoding process is time consuming, we explore Naive-Bayes based machine learning algorithm to accelerate VP9-to-HEVC transcoding. The machine learning process can bridge VP9 decoding information and HEVC splitting decision throughout training results, so the HEVC CU searching process can be simplified for faster transcoding. Two new features closely related to VP9, sub-block counting and depth map, are added for VP9-to-HEVC video transcoder. Using coding features and Naive-Bayes algorithm, several models are built for different QPs and block sizes to reduce transcoding time. Experiments show that a maximum time reduction of 53% and an average reduction of 44% can be achieved and the loss of decoded image quality in BD-rate is almost ignorable.

In this paper, a novel method of transcoding of HEVC encoded video is presented. The novel approach enables a very fast transcoding without the need for the image reconstruction. It is based on the principle of removing of the carefully selected coefficients from the bitstream. The removal of coefficients is done on the bitstream level and can easily be tailored to software as well as hardware implementations. The technique is able to provide a bitrate reduction in the order of up to 10%, with the image quality decrease of about 0.2–0.5dB.

Given a large-scale video archive of surgical interventions and a medical image showing a specific moment of an operation, how to find the most image-related videos efficiently without the utilization of additional semantic characteristics? In this paper, we investigate a novel content-based approach of linking medical images with relevant video segments arising from endoscopic procedures. We propose to approximate the video segments' content-based features by gradient-based signatures and to index these signatures with the Minkowski distance in order to determine the most query-like video segments efficiently. We benchmark our approach on a large endoscopic image and video archive and show that our approach achieves a significant improvement in efficiency in comparison to the state-of-the-art while maintaining high accuracy.

We present a framework for adaptive video encoding based on video content. The basic idea is to analyze the video to determine camera activity (tracking, stationary, or zooming) and then associate each activity with adaptive video quality constraints. We demonstrate our approach on the UT LIVE video quality assessment database that effective camera activity detection and classification is possible based on the motion vectors and the number of prediction units (PU) extracted using x265 HEVC encoding standard. In our results, by applying leave-one-out validation, we get an 79% correct classification rate using kNN binary classifier for the video segments.

It is important to generate both interesting and representative video summary for massive videos. This work proposes a new method to generate dynamic video summary using multiple features and image quality without human's involvement in the whole procedure. Specifically, we first split a video into several video clips. Second, a set of features including visual attention, exposure of light, saturation, hue, rule of thirds, contrast and directionality is computed and the qualities of video clips are also estimated. Then, the importance of each video clip is obtained based on these features and the estimated quality. Finally, based on the importance value, we sort clip values in descending order and select an optimal subset to generate video summary. Experimental results demonstrate that the proposed method enables to generate high-quality video summary.

Plenty of video stuff is created, broadcasted, shared and stored each and every day by industry experts, beginners, and hobbyists. Video summaries aim at showcasing the semantics and content of a clip in reduced time and space to enable a quick overview of video clip relevance. This paper focuses on static summaries showing key frames from the video. The key frames are extracted by leveraging the feature extraction and deep feature learning algorithm. The video summarization finds its application in semantic visualization, copy detection, indexing and browsing, concept detection, video services, video sharing and searching, video streaming, video caching, video quality assessment, and video transfer and video object tracking.

In many practical transcoding applications, such as video surveillance, the source videos are often contaminated by noise. The presence of noise not only results in poor compression efficiency and visual quality, but also imposes an adverse effect on the performance of subsequent video analysis tasks. Thereby it is very necessary to denoise the video. In this paper, we propose an efficient mode decision method for H.264 noisy video transcoding through analysing the effect of noise on mode decision. In the algorithm, we use the information available from previously decoded MBs to decide which modes can be overpassed with little loss to the rate-distortion performance. Experimental results show that our method saves the computational complexity nearly 65%, without noticeable rate-distortion(R-D) loss in comparison with the “Full-coding” (cascaded transcoding) method.

Video Watermarking is a prominent technique used to provide the authentication to the multimedia data such as audio, video and text. It is also used to resolve the copy right protection and ownership problem. The proposed method is used to provide the solution for copy right protection in video data. Here the different mathematical transforms like DWT and SVD are combined for the purpose of achieving the perceptual quality of the video and the key frame selection is achieved by chaotic map. Then the Two level DWT is performed in the video sequence, it produces the four sub bands like HH, HL, LH, LL. In this LL sub band is selected for embedding and extraction process. The watermark image is a binary image with proper resiolution. It is also subjected to SVD and performs the embedding process to embed the secret image into host video. This method survive different kinds of image and video processing attacks.

A high definition(HD) wide dynamic video surveillance system is designed and implemented based on Field Programmable Gate Array(FPGA). This system is composed of three subsystems, which are video capture, video wide dynamic processing and video display subsystem. The images in the video are captured directly through the camera that is configured in a pattern have long exposure in odd frames and short exposure in even frames. The video data stream is buffered in DDR2 SDRAM to obtain two adjacent frames. Later, the image data fusion is completed by fusing the long exposure image with the short exposure image (pixel by pixel). The video image display subsystem can display the image through a HDMI interface. The system is designed on the platform of Lattice ECP3-70EA FPGA, and camera is the Panasonic MN34229 sensor. The experimental result shows that this system can expand dynamic range of the HD video with 30 frames per second and a resolution equal to 1920∗1080 pixels by real-time wide dynamic range (WDR) video processing, and has a high practical value.

Capturing or acquiring video data from advanced gadgets, like cell phone cameras, smart phones, camcorders, etc., is very recent and up growing trend. With the advancement of the video camera as the integrated part of these multimedia devices generates necessity for video stabilization. Digital video stabilization or video frame alignment techniques are used to improve quality of input video by removing the undesired camera motions jitters; undesirable jiggles etc. Many video stabilization techniques are developed till now but only fast and low computational complexity approaches can be applied on the embedded devices. The paper presents the review of fast block matching local motion vector estimation and alignment algorithm. Effectiveness of the algorithm is shown by quantitative evaluation. The proposed work will also help to boost the performance of various error concealment, error resilience and video quality improvement techniques in H.264/HEVC video codec.

Virtual reality and 360-degree video streaming are growing rapidly; however, streaming 360-degree video is very challenging due to high bandwidth requirements. To address this problem, the video quality is adjusted according to the user viewport prediction. High quality video is only streamed for the user viewport, reducing the overall bandwidth consumption. Existing solutions use shallow buffers limited by the accuracy of viewport prediction. Therefore, playback is prone to video freezes which are very destructive for the Quality of Experience(QoE). We propose using layered encoding for 360-degree video to improve QoE by reducing the probability of video freezes and the latency of response to the user head movements. Moreover, this scheme reduces the storage requirements significantly and improves in-network cache performance.

Network traffic associated with video increases sharply, how to choose the interested information for a number of Internet users is challenging. So, technologies and applications related with video, such as video search, video fast browsing, video index and storage are in great demand. Behind these technologies and applications, a core problem is how to quickly browse massive video data and obtain the main content of the video. To solve this problem, different key frame extraction algorithms have been proposed. Due to the diversity of video content, different video have different characteristics. So the design of general video key frame extraction algorithm to solve the problem is not the reality. The main trend for the problem is to design the key frame extraction algorithm based on the characteristics of the video itself. In this article, we mainly focus on videos with edited boundaries and shot conversions. Aiming at this kind of video, we have designed and implemented video key frame extraction algorithm based on sliding window, the global feature Gist and local feature point detection algorithm SURF. In this algorithm, we use Gist feature to construct the global scene information of frames, and the SURF keypoint detection algorithm to extract local keypoints as local feature for each frame. Then, shot segmentation based on sliding window and shot merging algorithm is applied to dividing the original video into several shots. After that, we select the most representative frames in each video shot as key frames. Finally we evaluate the result of the algorithm from the subjective and objective perspective. Results show that key frames extracted in the algorithm are of high quality and can basically cover the main content of the original video.

Although Query-by-Example techniques based on Euclidean distance in a multidimensional feature space have proved to be effective for image databases, this approach cannot be effectively applied to video since the number of dimensions would be massive due to the richness and complexity of video data. The above issue has been addressed in two recent solutions, namely Deterministic Quantization (DQ) and Dynamic Temporal Quantization (DTQ). DQ divides the video into equal segments and extracts a visual feature vector for each segment. The bag-of-word feature is then encoded by hashing to facilitate approximate nearest neighbor search using Hamming distance. One weakness of this approach is the deterministic segmentation of video data. DTQ improves on this by using dynamic video segmentation to obtain varied-length video segments. As a result, feature vectors extracted from these video segments can better capture the semantic content of the video. To support very large video databases, it is desirable to minimize the number of segments in order to keep the size of the feature representation as small as possible. We achieve this by using only one video segment (i.e., no video data segmentation is even necessary) with even better retrieval performance. Our scheme models video using differential long short-term memory (DLSTM) recurrent neural networks and obtains a highly compact fixed-size feature representation with the output of hidden states of the DLSTM. Each of these features are further compressed by hashing them into binary bits via quantization. Experimental results based on two public data sets, UCF101 and MSRActionPairs, indicate that the proposed video modeling technique outperforms DTQ by a significant margin.

Although PRNU (Photo Response Non-Uniformity)-based methods have been proposed to verify the source camera of a non-stabilized video, these methods may not be adequate for stabilized videos. The use of video stabilization has been increasing in recent years with the development of novel stabilization software and the availability of stabilization in smart-phone cameras. This paper presents a PRNU-based source camera attribution method for out-of-camera stabilized video (i.e., stabilization applied after the video is captured). The scheme can (i) automatically determine if a given video is stabilized, (ii) calculate the fingerprint from a stabilized video, and (iii) effectively correlate the fingerprint computed from a stabilized video (i.e., anonymous video) with a fingerprint computed from another stabilized or non-stabilized video (i.e., a known video). Furthermore, experimental results show that the source camera of an anonymous non-stabilized video can be verified using a fingerprint computed from a set of images.

This paper introduces a new algorithm for improving the resolution of interpolated video. The aim is to achieve a high resolution video from a low resolution one. The new algorithm uses the concept of interpolation and filtering in order to get a higher resolution video. The filter is designed by reducing the size of the video and then enlarging it by using interpolation. A filter is optimized to minimize the mean square error between the original frame and the interpolated frame. Each frame will end up with a different filter. The video is then enlarged and the designed filters are used to sharpen the interpolated frames. The algorithm was successfully tested on different videos and the results were assessed subjectively and objectively by using the PSNR.

In the recent years the demand for real time video delivery has been increased. Hence, real time video distributers must ensure that the end-user quality-of-service is met (QoS). There are multiple conditions which can have influence on the transmission of video content, such as type of transmission, channel capacity, data loss, coding etc. Data loss mostly occurs in packet switched networks. This causes the encoder, on receiving side, to repeat the last good received frame, because of the lack of video stream. This is called video freezing issue or frame freezing, and it can appear in source of transmission and also in the receive side. Frame freezing effect is one of the common degradations in the quality of a video signals. Processing digital video in software can be computationally intensive task, even the simplest of algorithms can consume a great amount of memory. In this paper FPGA parallelism is exploited for implementation of video freezing detection algorithm.

No-reference video quality assessment can provide essential information for video service provides to improve user experiences. In this paper, we propose a no-reference video quality assessment method by utilizing the hybrid parameters extracted from compressed video frame. In particular, the proposed method, namely FRAME-FEBP(FRAME-Feature Extraction in Bit stream & Pixel), models the video quality by using both the high-level syntax elements from bit-stream and the statistics calculated on pixels. The experiments show that the proposed method can provide satisfactory results on video quality assessment.

In this article deals with estimating and analysis capacity of the closed video channel in the case of applying the method for processing video data based on standardized MPEG technology, the method for closing the video based on the series circuit (compression followed by encryption), the method to hide all the video after the DCT blocks of the base of the video frame, the method based on the selection of significant structural units of the base video frame.

The requirement for real-time video transcoding systems has significantly increased due to the easy and widely available access to high resolution video streams and large-scale applications in recent years. In this paper, we propose a real-time distributed video transcoding system working on heterogeneous environment to tackle the high requirement of such applications. It allows multiple computers to be networked together to execute the same transcoding task so that the system can process more video streams in real time. Most importantly, the proposed method emphasizes on the velocity of the video data which involves the continuous input video stream and outcomes of transcoded video output stream that is accessible on-the-fly in contrast to the batch-oriented approach such as the MapReduce framework, where output latency can be significant. The performance of the proposed system can be further improved by using a more intelligent scheduler for video frames distribution.

Automated mensuration of objects in scenes can be used as a strategy for automating the assessment of video quality. The Video National Imagery Interpretability Rating Scale (VNIIRS) provides criteria for rating the quality of video according to the level of detail that humans can detect in moving objects. To measure the level of detail in video frames automatically for estimating VNIIRS, commercial off-the-shelf tracking software is used to identify moving objects. A supervised learning algorithm produces segmentation of video frames, and vehicles and people are measured using an automated algorithm to find the size of the targets identified by the moving object detector. The measurements provide an estimate of the ground sampling distance of each video frame. This system of mensuration can be used to rate video quality based on in-scene information only, without relying on source metadata.

Non-Photorealistic Rendering (NPR) for video sequences can be regarded as an extension of NPR for images. However, if the NPR sequence is obtained from a real video using an image processing pipeline, discontinuity problems have to be addressed, especially for the background region. In this paper, we propose to adopt Fast Video Segmentation algorithm to extract the common background in the video before applying NPR. With the help of the moving object mask, the background and moving objects in the scene can be rendered respectively, and combined together to generate final NPR sequences with better qualities.

We propose a High-Definition Multimedia Interface (HDMI) video scaling system based on an all-programmable SoC, which can be used in any image processing system requiring a two-dimensional (2D) interpolation accelerator. This structure work on a platform-based SoC environment. By the AXI Video Direct Memory Access (AXI VDMA), video data are directly transferred between DDR3 memory and the built-in FPGA where the proposed buffering scheme and interpolation accelerator are implemented. We store video data in the DDR3 memory and the interpolation accelerator performs real-time video scaling. An experiment with HDMI video input from a Microsoft Windows PC and HDMI video output to another display proves the effectiveness of the real-time video scaling.

Video copy detection is still an open problem as current approaches are not able to carry out the detection with enough efficacy and efficiency. These are desirable features in modern video-based applications requiring real-time processing in large scale video databases and without compromising detection performance, especially when facing non-simulated video attacks. These characteristics are also desirable in partial-copy detection, where the detection challenges increase when the video query contains short segments corresponding to a copied video, this is, partial-copies at frame level. Motivated by these issues, in this work we propose a video fingerprinting approach based on the extraction of a set of low-cost and independent binary global and local fingerprints. We tested our approach with a video dataset of real-copies and the results show that our method outperforms robust state-of-the-art methods in terms of detection scores and computational efficiency. The latter is achieved by processing only short segments of 1 second length, which takes a processing time of 44 ms.

In this work, a single input multiple output (SIMO) transcoding architecture is proposed. SIMO will benefit the mobile edge computing (such as HTTP Live Streaming requiring multiple copies of the video streams at different quality levels) without resorting to the legacy transcoding that video stream is completed decoded and encoded multiple times without exploring the compressed information. Leveraging the information encoded in the existing video streams, we could reduce the search candidates when transcoding the high quality bitstream to other versions with reduced quality level. As the first step, we have demonstrated the SIMO idea with bit rate shaping (i.e., bit rate transcoding) only scenario. It has shown more than 2x complexity reduction without quality loss using the common test conditions.

In this paper, we pose a new problem of video enhancement transcoding, which converts the compressed dark video into compressed normal-lighting one. Distinct statistics of dark and normal videos result in quite different coding modes, which thus enforces latent constraints on mode conversion during transcoding. Following this idea, we propose a fast mode decision algorithm to speed up computation while maintaining rate-distortion (RD) performance. Experimental results show that our method saves the computational complexity nearly 70%, without noticeable RD loss in comparison with the cascaded decoder-encoder approach.

The acquisition rate of video equipment makes positive advances, for example, a video with Full-HD resolution can be acquired at 1000 Hz. However, the acquisition process in imaging system is independent from video encoding process. This means that the imaging system does not consider to reduce video encoding rate. This paper proposes a video generation algorithm optimized in terms of video encoding. The proposed method features to use a video signal sampled at high temporal resolution. The video signal sampled at high temporal resolution contains much information of captured scene. Utilizing such information for generating video signal, we can pioneer a way to improve coding efficiency outside the conventional research area of video coding. The proposed method applies a temporal filter that features to select filter coefficients from the candidate set. The selection of filter coefficients for each frame is designed based on dynamic-programming in order to minimize total encoding rate of all generated frames. Experimental results show that the proposed method can reduce encoding rate on average by 2.14 [%] compared to constant mean filter.

Rapid development of up-to-date information technologies and the advent of the Web have accelerated the growth of digital media and, in particular, video collections. Due to semantic gap between the low-level video features and high-level interpretations lots of difficulties remain in the construction of video stream semantic structure. Relational model of video parsing has been proposed. Each frame segmentation lies on the basis of on line search for boundaries of homogeneous consecutive frames presenting an unit for semantic analysis.

In order to extract the horizontal synchronization information of stochastic video signal and reconstruct video image stably and automatically, the computer video signal captured from the video cable will be analyzed and the method of correlation of stochastic video signal is put forward in this paper. Without synchronization information and resolution information of the monitors, the video images are reconstructed by calculating the correlation of video signals, and the reconstructed images are amended by further processing. The experiment results can benefit stochastic video signal processing and application in video industrial production.

Smart video analysis is attracting increasing attention with the pervasive use of surveillance camera. In this paper, we address video anomaly detection by Uniform Local Gradient Pattern based Optical Flow (ULGP-OF) descriptor and one-class extreme learning machine (OCELM). Using the proposed ULGP-OF descriptor, we naturally combine the robust 2D image texture descriptor LGP with video optical flow to jointly descibe the texture and motion characteristics of video. ULGP-OF significantly outperforms other frequently-used classic video decriptors by a 6% to 10% EER reduction. As to normal video event modeling, the newly emergent ELM is introduced for the first time to tackle the unbearable training time incurred by massive training data from video streams. Compared to classic data description algorithms like one-class SVM (OCSVM) and sparse coding, OCELM can yield competitive results with a significant improvement in learning speed, which makes our approach more applicable to large-scale video analysis and easier for updating when video data are explosively generated in this day and age. Moreover, by adopting consistency-based criteria, only one parameter needs to be appointed for OCELM before training, which renders our approach much more parameter-free than other anomaly detection techniques like sparse coding. Experiments on UCSD ped1 and ped2 datasets demonstrate the effectiveness of our approach.

View synthesis technique for 3D video and free viewpoint video (FVV) using existing view(s) can avoid the large volume of video data transmission. Existing techniques may concern poor rendering quality by missing pixel values (i.e. creating holes) due to the occluded region, rounding error and disparity discontinuity. To address those problems with the existing techniques uses correlations in spatial texture only or both spatial texture and temporal background. The former techniques (e.g. inpainting) suffer quality degradation due to lack of spatial correlation on the foreground-background boundary areas. The latter techniques (e.g. background update with Gaussian Mixture-based Modelling (GMM)) can improve quality in some occluded areas, however, due to the dependency on warping of background image and spatial correlation they still suffer quality degradation. In this paper, we propose a new hole-filling technique using the number of GMM model rather than the background image to identify background/foreground pixels. The missing pixels of background and foreground are recovered from the background pixel and the weighted average of warped and foreground model pixels respectively. The experimental results show that the proposed approach provides 0.9∼1.7dB PSNR improvement compare to the state-of-the-art method.

This paper proposed a video frame skipping and interpolation technique (VFS-I) for streaming video. This technique is divided into main sections: transmitter and receiver. The transmitter is employed FSM (Frame Skipping Mechanism) to drop/skip similar frames from video sequence to reduce the amount of frames that need to be transferred over the network. This method specified frames to-be-kipped according to motion estimation between every neighboring frame. After that, re-encode the remaining video frames, and sent it to the receiver. The receiver employed RFI (Reconstructing video-Frames Interpolation) to rebuild the frames that were dropped in the transmitter side. The motion compensated interpolation of the key-frame (remaining frame) is used to reconstruct video frames back. A unified design of frame-skipping and interpolation mechanism are made for incrementing and improving compression process efficiency that have not considered in the previous works. One of the main advantages of the VFS-I is characterized by flexibility and scalability of video streaming to the requirements of the network bandwidth and end user.

In this paper, we present a simple but effective video watermarking scheme robust enough against scalable lossy recompression and transcoding. Most of video watermarking algorithms use DCT or DWT, but few of those method can withstand scalable recompression or transcoding. Algorithms with DWT and DCT usually based on specific encoding format, so the watermark will be lost after transform embedded video to other encoding formats. In our method, we use scene change to choose frames to embed watermark, so it can resist attack of aimless frame dropping. We use Arnold transformation to enhance the imperceptibility of the watermark. Our watermark is a fixed-size binary image, after Arnold transformation, it is uniformly embedded to the chosen frames by a spatial random mapping algorithm. The watermark after embedding to the frame has a high imperceptibility. The proposed algorithm allows blind detection without use of origin video. The embedded watermark can be extracted in a low resolution video recompressed from an embedded high resolution video or transcoded from other format embedded video.

Recurring visual elements in videos commonly represent central content entities, such as main characters and dominant objects. The automated detection of such elements is crucial for various application fields ranging from compact video content summarization to the retrieval of videos sharing common visual entities. Recent approaches for content-based video analysis commonly require for prior knowledge about the appearance of potential objects of interest or build upon a specific assumption, such as the presence of a particular camera view, object motion, or a reference set to estimate the appearance of an object. In this paper, we propose an unsupervised, model-based approach for the detection of recurring visual elements in a video sequence. Detected elements do not necessarily represent an object, yet, they allow for visual and semantic interpretation. The experimental evaluation of detected models across different videos demonstrate the ability of the models to capture potentially high diversity in the visual appearance of the traced elements.

Video cataloging is the main technology of video management and reuse. It helps to quickly and accurately query tapes or films and improves work efficiency greatly. However, current cataloging is performed manually because existing computer-aided cataloging system can't achieve expected accuracy and speed. In this paper, it proposes an algorithm to control video structure analysis process cleverly. For video shot boundary detection, it put forwards Poisson model based inter-frame difference histogram to generate threshold. For video scene boundary detection, it raises variable-length sliding window to control detection process. It dynamically adjusts the length of sliding window to avoid too much or too little scene segmentations. Experiments for video shot and scene boundary detection show its efficiency in increasing detection accuracy.

The depth image based rendering (DIBR) plays a key role in 3D video synthesis, by which other virtual views can be generated from a 2D video and its depth map. However, in the synthesis process, the background occluded by the foreground objects might be exposed in the new view, resulting in some holes in the synthetized video. In this paper, a hole filling approach based on background reconstruction is proposed, in which the temporal correlation information in both the 2D video and its corresponding depth map are exploited to construct a background video. To construct a clean background video, the foreground objects are detected and removed. Also motion compensation is applied to make the background reconstruction model suitable for moving camera scenario. Each frame is projected to the current plane where a modified Gaussian mixture model is performed. The constructed background video is used to eliminate the holes in the synthetized video. Our experimental results have indicated that the proposed approach has better quality of the synthetized 3D video compared with the other methods.

Development of image stitching techniques, whichtake multiple images and stitch them together to make naturallooking panoramas, is an integral part of the new wave invisual media -- the 360 surround displays, such as the OculusRift. However, as compared to static image stitching, there hasbeen little research conducted on video stitching, especiallymethods of generating panoramic video content from a singlemoving video camera. In this paper we present an effectivemeans of creating video panorama from a single camera. Ourvideo stitching method allows for widening the field of view ofa video while maintaining spatial-temporal consistency. In thisapproach, a single image warping transformation, orhomography, is obtained by averaging several rotationmatrices from several frames. The application of the samehomography to every frame, with fixed camera intrinsicparameters, provides enhanced stability of the resultingpanoramic video. This novel stitching method can be used for avariety of applications, such as allowing the creation ofpanoramic video with smartphones and enabling gaming andmovie companies to make panoramic video content in a flexibleand inexpensive manner.

The emergence of wearable devices such as portable cameras and smart glasses makes it possible to record life logging first-person videos. Browsing such long unstructured videos is time-consuming and tedious. This paper studies the discovery of moments of user's major or special interest (i.e., highlights) in a video, for generating the summarization of first-person videos. Specifically, we propose a novel pairwise deep ranking model that employs deep learning techniques to learn the relationship between high-light and non-highlight video segments. A two-stream network structure by representing video segments from complementary information on appearance of video frames and temporal dynamics across frames is developed for video highlight detection. Given a long personal video, equipped with the highlight detection model, a highlight score is assigned to each segment. The obtained highlight segments are applied for summarization in two ways: video time-lapse and video skimming. The former plays the highlight (non-highlight) segments at low (high) speed rates, while the latter assembles the sequence of segments with the highest scores. On 100 hours of first-person videos for 15 unique sports categories, our highlight detection achieves the improvement over the state-of-the-art RankSVM method by 10.5% in terms of accuracy. Moreover, our approaches produce video summary with better quality by a user study from 35 human subjects.

Visual Quality Assessment of 3D/stereoscopic video (3D VQA) is significant for both quality monitoring and optimization of the existing 3D video services. In this paper, we build a 3D video database based on the latest 3D-HEVC video coding standard, to investigate the relationship among video quality, depth quality, and overall quality of experience (QoE) of 3D/stereoscopic video. We also analyze the pivotal factors to the video and depth qualities. Moreover, we develop a No-Reference 3D-HEVC bitstream-level objective video quality assessment model, which utilizes the key features extracted from the 3D video bitstreams to assess the perceived quality of the stereoscopic video. The model is verified to be effective on our database as compared with widely used 2D Full-Reference quality metrics as well as a state-of-the-art 3D FR pixel-level video quality metric.

The multi-view video streaming is one of promising technologies for emerging video services. For those applications of multi-view video streaming, video frames of all viewpoints, i.e., cameras, are needed to be transmitted to viewers because the demands of all the viewers' view-switching are unpredictable. However, existing transmission schemes are highly vulnerable to frame loss. Specifically, the frame loss in one viewpoint induces a collapse of decoding for other viewpoint videos. To improve loss-resilience, we propose a multi-path based multi-view video transmission. Our scheme encodes video frames into multiple descriptions that are mutually independent of each other, by using inter-view prediction. It then transmits each description using multiple transmission paths. Our scheme makes three contributions:1) it reduces video traffic even for a large number of cameras, 2) it prevents an increase in the number of undecoded video frames caused by one frame loss, and 3) it conceals frame loss by using the video frames in the other paths. Our scheme generalizes to an arbitrary number of transmission paths and discusses the detailed performance at the number of the paths of 2. Evaluations show that our proposed scheme improves video quality by 3 dB compared to existing transmission schemes in loss-prone environments.

Soft video delivery, i.e., analog video transmission, has been proposed to provide graceful video quality in unstable wireless channels. However, existing analog schemes need to transmit a significant amount of metadata to a receiver for power allocation and decoding operations. It causes large overheads and quality degradation because of rate and power losses. To reduce the overheads while keeping high video quality, we propose a new analog transmission scheme. Our scheme exploits a Gaussian Markov random field for modeling video sequences to significantly reduce the required amount of metadata, which are obtained by fitting into the Lorentzian function. Our scheme achieves not only reduced overhead but also improved video quality, by using the fitting function and parameters for metadata. Evaluations using several test video sequences demonstrate that our proposed scheme reduces overheads by 97 % with 3.4 dB improvement of video quality compared to the existing analog video transmission scheme.

Video surveillance has been widely used in many applications. Public safety and theft protections are most important uses of it. A system like this needs an efficient transmission and storage of the large video data. Key frame extraction is a simple and powerful system to accomplish this objective. Keyframe extraction also called as a summary of the video because it gives the only important content of the video. Keyframing is used to summarize essential video content, hence it is very important for video data applications specially for surveillance where a lot of redundancy is present. Keyframes can be determined by considering motion as an important feature which can be calculated by using inter frame difference. Proposed method uses a correlation of colour channels to summarize video content.

Synchronizing videos over file-hosting services on personal cloud such as Dropbox, Box or Onedrive leads to wastage in bandwidth and storage, which can be critical, while using mobile devices. Users can alternatively download the video on-the-go, but that leads to high latency, depending on network bandwidth and video file size. In contrast, adaptive video streaming allows near-real-time viewing by streaming the best possible quality in a given network condition. This feature is achieved by keeping multiple versions of video in cloud, leading to additional costs in cloud storage. Moreover, current solutions can only support a small set of bitrates, leading to abrupt switches in video resolution especially when the network condition is unstable, as often experienced by mobile users. This paper introduces Vsync, a framework for cloud based video synchronization for mobile devices. A video content is streamed using a cloud-based real-time transcoding and transmission framework to provide smooth video quality. Built over prediction models for video transcoding sessions and a QoE based adaptive video streaming protocol, Vsync is able to obtain the improvements of 37 ∼ 80% than other compared schemes. The dataset and evaluation was done on a pool of 220K video clips.

Nowadays mobile devices have more processing power and are capable of performing complex tasks such as processing multimedia data. Free-viewpoint TV is one such complex task, where the user can choose a custom viewpoint from where to view a scene. In this work, the depth-image-based rendering (DIBR) technique is used to generate this virtual view by utilizing the Graphical Processing Unit (GPU) of the mobile device. The limited resources of the device were taken into consideration when designing the system. Experimental results show that the maximum performance achieved for a 1024 × 768 resolution video is 19.17 frames per second and the best Peak Signal-to-Noise Ratio (PSNR) value for this resolution is 35.09 dB.

There is tremendous growth in the amount of digital video creation and communication with the proliferation of mobile consumer electronic devices such as cellular phones with video capture and display capabilities. This trend will continue with the demand for higher spatial and temporal video resolutions accompanied by 3D and multiview video. Since it is not practical to store or transmit the raw digital video data, it is essential to reduce it with digital video compression. To cope with the vast amount of raw digital video data, digital video compression standards use increasingly more complex algorithms. These algorithms achieve higher compression at the same quality at the expense of significant computational complexity. Therefore, it is essential to design low power digital video compression hardware especially for mobile devices. In this talk, I will present two sets of power reduction techniques for digital video compression hardware. The ones in the first set reduce the power consumption with no impact on the video quality. The ones in the second set trade off video quality for power consumption which also enables adjusting video quality based on available energy in the battery of a mobile device for graceful degradation. I will show the impact of these techniques on the power consumption of digital video compression hardware implementing H.264/AVC and H.265/HEVC standards.