Search

We propose a 2.5D video editing system called DDMixer2.5D. 2.5D video contains not only color channels but also a depth channel, which can be recorded easily using recently available depth sensors, such as Microsoft Kinect. Our system employs this depth channel to allow a user to quickly and easily edit video objects by using simple drag-and-drop gestures. For example, a user can copy a video object of a dancing figure from video to video simply by dragging and dropping using finger on the touch screen of a mobile phone handset. In addition, the user can drag to adjust the 3D position in the new video so that contact between foot and floor is preserved and the size of the body is automatically adjusted according to the depth. DDMixer2.5D has other useful functions required for practical use, including object removal, editing 3D camera path, creating of anaglyph 3D video, as well as a timeline interface.

User-generated music video clip called MAD movie, which is a derivative (mixture or combination) of some original video clips, are gaining popularity on the web and a lot of them have been uploaded and are available on video hosting web services. Such a MAD music video clip consists of audio signals and video frames taken from other original video clips. In a MAD video clip, good music-to-image synchronization with respect to rhythm, impression, and context is important. Although it is easy to enjoy watching MAD videos, it is not easy to generate them. It is because a creator needs high-performance video editing software and spends a lot of time for editing video. Additionally, a creator is required video editing skill. DanceReProducer (Nakano et al [2011]) is a dance video authoring system that can automatically generate dance video appropriate to music by reusing existing dance video sequences. It trains correspondence relationship between music and video. However, DanceReProducer cannot train video sequence information because it only trains one-bar correspondence relationship. So we improved DanceReProducer to consider video sequence information by using Markov chain of latent variable and Forward Viterbi algorithm.

A transrating-assisted spatial resolution conversion method for compressed videos is proposed. By utilizing the property of equivalence between downsampling of the video spatial resolution and bitrate reducing of the video streams in frequency domain, it implements video resizing through transrating. That is, it decodes the compressed video stream and then reencodes it to a new bitrate corresponding to the target video spatial resolution instead of direct changing the video spatial resolution before the conversion like the traditional spatial transcoding, which facilitates efficient using of motion vectors and macroblock modes. To alleviate mismatching between the original bitrate allocation mechanism and the newly-changed distribution of the discrete cosine transform(DCT) coefficients, a tactic which only proactively sets the high frequency DCT coefficients of P frames to zeros is used in the transrating process to implicitly implement subsampling of the video. Experimental results show that it can significantly improve the rate-distortion performance of the obtained video streams.

With the entry of 3G services, video based applications like live video streaming, gaming applications, video telephony are going to be in great demand. The data speed is limited in Mobile applications as the bandwidth available is a limited resource. The demand for video streaming is expected to grow in multi folds in future and speed is a critical issue in video streaming. Techniques that reduce the video bit rate are the best way to have a smooth video streaming for a given bandwidth. The mobile equipment screen displays are usually small in size, this size can hence be exploited to have a certain level of compromise with video resolution and by deploying good video compression techniques, the bit rate can be reduced to suit the mobile based applications. Improved video compression is an example of better spectral efficiency (getting the most out of limited bandwidth), which is what makes 3G cellular technology more advanced than its predecessors.

Due to the recent success of 3D cinema, 3D video has been gathering considerable interest and delivery of 3D video to home TVs and mobile devices has been actively researched. A broadcast-friendly 3D video delivery format is so-called 'view plus depth', where single video sequence is augmented by dense depth information. Based on it, desired views can be synthesized at the receiver side. While well susceptible for effective compression, this representation presents problems related with occluded areas which become visible in the synthesized views. Different occlusion filling techniques have been developed varying in quality and complexity. This contribution aims at providing an objective and subjective comparison of three such techniques. These include adaptive pre-filtering of depth maps, and one simplified and one high-performance inpainting techniques. The latter two have been specifically modified for the case of occlusion filling. We have aimed especially at ranking the performance in the case of mobile 3D imaging. Along with objective comparisons, results from subjective tests are presented as well. Both groups of results demonstrate the superiority of the second inpainting technique, which is however also the most computationally expensive one.

The high availability of video streams is making necessary mechanisms for indexing such contents in the Web world. In this paper we focus on news programs and we propose a mechanism that integrates low and high level video features to provide a high level semantic description. A color/luminance analysis is coupled with audio analysis to provide a better identification of all the video segments that compose the video stream. Each video segment is subject to speech detection and is described through MPEG7 so that the resulting metadata description can be used to index the video stream. An experimental evaluation shows the benefits of integrating audio and video analysis.

Video data is stored and distributed in large quantities and a wide variety of formats. As the amount of material increases, so too does the need to manage and search it. In previous work, we addressed the problem of rapidly locating a given clip in a large stream of video by analysing the pattern of edits or cuts, a method that is fast and as effective as a standard search method based on image comparison, but is not very robust, particularly for short clips. In this paper we describe two new methods that address the limitations of the cut-based approach. We show that the new video representations are more robust than any previous method and vastly superior at identifying similar content when the cut-based signature is not sufficiently discriminatory.

This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying stream of images. While the individual frames of a video texture may be repeated from time to time, the video sequence as a whole is never repeated exactly. Video textures can be used in place of digital photos to infuse a static image with dynamic qualities and explicit actions. We present techniques for analyzing a video clip to extract its structure, and for synthesizing a new, similar looking video of arbitrary length. We combine video textures with view morphing techniques to obtain 3D video textures. We also introduce video-based animation, in which the synthesis of video textures can be guided by a user through high-level interactive controls. Applications of video textures and their extensions include the display of dynamic scenes on web pages, the creation of dynamic backdrops for special effects and games, and the interactive control of video-based animation.

The rapidity with which digitat information, particularly video, is being generated, has necessitated the development of tools for efficient search of these media. Content based visual queries have been primarily focussed on still image retrieval. In this papel; we propose a novel, real-time, interactive system on the Web, based on the visual paradigm, with spatio-temporal attributesplaying a key role in video retrieval. We have developed algorithms for automated video object segmentation and tracking and use real-time video editing techniques while responding to user queries. The resulting system pe$orms well, with the user being able to retrieve complex video clips such as those of skiers, baseball players, with ease.

This paper describes a technique that transforms ordinary video segments into animations that have a hand-painted look. Our method is the first to exploit temporal coherence in video clips to design an automatic filter with a hand-drawn animation quality, in this case, one that produces an Impressionist effect. Off-the-shelf image processing and rendering techniques are employed, modified and combined in a novel way. This paper proceeds through the process step by step, providing helpful hints for tuning the off-the-shelf parts as well as describing the new techniques and bookkeeping used to glue the parts together.

Video data offers users of multimedia systems a wealth of information; however, it is not as readily manipulated as other data such as text. Raw video data has no immediate “handles” by which the multimedia system user may analyze its contents. To assist human analysis of digital video data, a system has been developed to perform automatic, contentbased video indexing from object motion. The AVI (Automatic Video Indexing) System allows users to specify queries on video sequences and “jump” to points of interest to avoid tedious serial playback. For example, the user may select a person in a video sequence and specify the query “show me all objects that this person removed from the scene”. In response, the AVI system assembles a set of video “clips” highlighting the query results. The user may select a clip of interest and proceed with further video analysis using playback or additional queries.

Efficient indexing and retrieval of digital video is an important aspect of video databases. One powerful index for retrieval is the text appearing in them. It enables content- based browsing. We present our methods for automatic segmentation and recognition of text in digital videos. The algorithms we propose make use of typical characteristics of text in videos in order to enable and enhance segmentation and recognition performance. Especially the inter-frame dependencies of the characters provide new possibilities for their refinement. Then, a straightforward indexing and retrieval scheme is introduced. It is used in the experiments to demonstrate that the proposed text segmentation and text recognition algorithms are suitable for indexing and retrieval of relevant video scenes in and from a video data base. Our experimental results are very encouraging and suggest that these algorithms can be used in video retrieval applications as well as to recognize higher semantics in video.

This paper presents an integrated solution for computer assisted video parsing and content-based video retrieval and browsing. The uniqueness and effectiveness of this solution lies in its use of video content information provided by a parsing process driven by visual feature analysis. More specifically, parsing will temporally segment and abstract a video source, based on low-level image analyses; then retrieval and browsing of video will be based on key-frames selected during abstraction and spatial-temporal variations of visual features, as well as some shot-level semantics derived from camera operation and motion analysis. These processes, as well as video retrieval and browsing tools, are presented in detail as functions of an integrated system. Also, experimental results on automatic key-frame detection are given.

The Personal Presence System (PPS) experimental prototype is being designed to support multiparty multimedia visual services which use advanced video combining techniques. This paper is a companion to another paper in this proceedings: “The Personal Presence System—A Wide Area Network Service Resource for the Real Time Composition of Multipoint Multimedia Communications” which contains a detailed service description.This paper describes the architecture of the Advanced Video Bridge (AVB) (Fig. 1) experimental prototype that is central to PPS. This bridge is a shared resource designed for deployment in proximity to other switching and transcoding resources. It provides a single point of contact for each user so that only one combined video stream is received. This solves the N squared transmission problem of multipoint video conferencing and allows customer premises terminal equipment to be much simpler and less expensive to promote universal video teleconferencing.The AVB is made of a multitude of atomic modules called Video Composing Modules (VCMs) (Fig. 5) which are linked together on an as needed basis to form Video Composing Chains (VCCs) (Fig. 4) of variable length. Each conference participant uses one VCC to customize his or her view of the other participants and conference video objects. Each user has complete and individual control over the appearance and associative connectivity of their video display.The unprecedented degree of control given to each user by the PPS-AVB is due to the nature of the VCM which utilizes the mechanism of pel rate priority multiplexing to give the customer the now familiar paradigm of many overlapping windows that they can move, grow, and shrink at will. The VCM also provides the abilities to temporally subsample or freeze video frames, and to cut out portions of a video scene from the background using rectangular frames and/or keying based upon the color and brightness of objects. The AVB will also be a universal converter of standards and coding schemes so that people with different terminal equipment, having different scan rates and formats and codings can talk to each other without barriers.

The data driven, bottom up approach to video segmentation has ignored the inherent structure that exists in video. This work uses the model driven approach to digital video segmentation. Mathematical models of video based on video production techniques are formulated. These models are used to classify the edit effects used in video and film production. The classes and models are used to systematically design the feature detectors for detecting edit effects in digital video. Digital video segmentation is formulated as a feature based classification problem. Experimental results from segmenting cable television programming with cuts, fades, dissolves and page translate edits are presented.

This paper introduces a set of techniques for processing video data compressed using JPEG compression at near real-time rates on current generation workstations. Performance is improved over traditional methods by processing video data in compressed form, avoiding compression and decompression and reducing the amount of data processed. An approximation technique called condensation is developed that further reduces the complexity of the operation. The class of operations that are computable using the techniques developed in this paper are called linear, global digital specials effects (LGDSEs), and represent those effects where a pixel in the output image is a linear combination of pixels in the input image. Many important video processing problems, including convolution, scaling, rotation, translation, and transcoding can be expressed as LGDSEs.

A new approach to interacting with stored video is proposed. The approach utilizes VideoMAP and VideoSpaceIcon. VideoMAP is the interface that shows the essential video features in an easy to perceive manner. VideoSpaceIcon represents the temporal and spatial characteristics of a video shot as an intuitive icon. A video indexing method supports both tools. These tools allow the user's creativity to directly interact with the essential features of each video by offering spatial and temporal clues. This paper introduces the basic concept and describes prototype versions of the tools as implemented in a video handling system. VideoMAP and VideoSpaceIcon are effective for video handling functions such as video content analysis, video editing, and various video applications which need an intuitive visual interface.