Search

With the development of the broadband network, video communications such as videophone, video distribution, and IPTV services are beginning to become common. In order to provide these services appropriately, we must manage them based on subjective video quality, in addition to designing a network system based on it. Currently, subjective quality assessment is the main method used to quantify video quality. However, it is time-consuming and expensive. Therefore, we need an objective quality assessment technology that can estimate video quality from video characteristics effectively. Video degradation can be categorized into two types: spatial and temporal. Objective quality assessment methods for spatial degradation have been studied extensively, but methods for temporal degradation have hardly been examined even though it occurs frequently due to network degradation and has a large impact on subjective quality. In this paper, we propose an objective quality assessment method for temporal degradation. Our approach is to aggregate multiple freeze distortions into an equivalent freeze distortion and then derive the objective video quality from the equivalent freeze distortion. Specifically, our method considers the total length of all freeze distortions in a video sequence as the length of the equivalent single freeze distortion. In addition, we propose a method using the perceptual characteristics of short freeze distortions. We verified that our method can estimate the objective video quality well within the deviation of subjective video quality.

Architecture of an IP-based narrow-band videophone system is proposed in this paper for convenient videophone calls between any two computers even if being placed in two different LANs within network agents. The bandwidth need of each call is less than 256 kbps. The system consists of two kinds of entities: Videophone Terminals (VPT) and a Video Call Server (VCS). A VPT is actually a microcomputer program, composed of 4 primary parts, an audio codec, a video codec, a media deliverer/receiver and a call controller. The basic functions of the VCS include videophone number generation and management, access admission and address resolution. The VCS with a public IP address plays an important role in the system especially when a video call has to penetrate through network agents. Each VPT in the system gets its own external transport address from the VCS through registration process. A calling VPT would receive the external transport address of the called VPT from the VCS through address resolution. The proposed system works and is helpful to accelerate the realization of people's videophone dream over IP networks.

In low bit-rate or very low bit-rate video applications, such as videophone and video conferencing, it is difficult for traditional methods to gain a satisfactory image quality. They often suffer from some shortcomings: images with artifacts caused by limited bandwidth and motion discontinuity caused by frame skipping when bit-streams are transmitted through a fixed rate channel in low-delay applications. To attack the two problems above, we propose a content-based rate control method for videophone and video conferencing applications. Our work is focused on three aspects: analysis of bit-streams in low bit-rate video applications, content-based bit allocation to facial areas and background, and low-delay rate control in object-layer. Experiments show that the proposed method can achieve better subjective quality compared with that of the test model TMN5 of H.263, and slightly worse performance in PSNR, but with significantly better subjective quality in the facial areas, compared with that of the TMN8 of H.263+.

The transmission of real-time images and video over wireless communication channels is still a challenge problem. Digital compressed images are sensitive to bit errors which are typical in wireless communications. Moreover, the bandwidth at the air interface is currently a limiting factor because the first and second generation of mobile phone standards mainly support voice communications. In this paper, we present our study of real-time image traffic over a radio link -- the aim of this research is for videophone applications. In this study, we use the Discrete Wavelet Transform (DWT) to compress images and a Code Division Multiple Access (CDMA) link to transfer images over wireless communication channels. The results of the experiment show that it is possible to transfer 4 QCIF images per second over a CD MA link with minor degradation in image quality. This study was investigated by the VLSI Signal, Image and Video Processing Research Laboratory at the University of California, San Diego (UCSD).© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper presents a low bit rate videophone system for deaf people communicating by means of sign language. Classic video conferencing systems have focused on head and shoulders sequences which are not well-suited for sign language video transmission since hearing impaired people also use their hands and arms to communicate. To address the above-mentioned functionality, we have developed a two-step content-based video coding system based on: (1) A segmentation step. Four or five video objects (VO) are extracted using a cooperative approach between color-based and morphological segmentation. (2) VO coding are achieved by using a standardized MPEG-4 video toolbox. Results of encoded sign language video sequences, presented for three target bit rates (32 kbits/s, 48 kbits/s and 64 kbits/s), demonstrate the efficiency of the approach presented in this paper.

This paper presents a layered video-coding scheme [1] for very low bit rate videophone. The lowertwo layers (the first and second layer) of this scheme are compatible with ITU-T H.263 [2] in order to becompatible with other schemes. The first layer (basic layer) is normal H.263 withouta priori knowledgeof the image content, which is robust to image content. The second layer introduces the typical head-shoulder assumption, and provides subjectively improved quality of reconstructed image. The third layeris based on the same model as layer two, but further improves the subjective quality by employing theshape information of the interested objects. Manipulation functionality is supported in the third layer.When the typical head-shoulder hypothesis is true, the coding scheme can upgrade its coding layer toobtain better subjective quality. On the other hand, the coding scheme can withdraw to robust lowerlayers when the a priori knowledge is not available. In this paper, a novel motion estimation criterion isintroduced which can be integrated into most of present strategies to improve the overall performance. AB-spline approach is introduced in this paper to encode the object contour instead of chain code.Keywords: layered-coding, object-based, contour-coding, arbitrary-shaped, motion estimation criteria

A suite of bandwidth efficient image codecs are presented for the use in second-generation wireless systems, such as the American IS-54 and IS-95 systems, the Pan-European GSM system and the Japanese digital cellular system. The proposed codecs are configured to operate at 9.6K bits per second and are suitable for Quarter Common Intermediate Format videophone sequences, scanned at 10 frames per second. The new image codecs employ the orthonormal wavelet transform to decompose the displaced frame difference data for each frame, into four frequency subbands. The wavelet coefficients within the frequency subbands are then encoded using vector quantization. Comparison measures are undertaken for the two-stage pairwise nearest neighbor (PNN) algorithm and the designs are rated upon their ability to coherently reconstruct an efficient codebook from a training sequence of vector coefficients. It was found that the two- stage PNN algorithm constitutes a valuable compromise in terms of computational complexity with only negligible performance loss. When the codecs were configured to operate at 9.6Kpbs, the average peak signal to noise ratio of the two stage PNN and the adaptive algorithms were in excess of 28kB and 30dB respectively.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This research addresses the use of face segmentation to improve the subjective quality of videophone sequence encoded by the H.261-compliant coder. In this approach, each frame of the input video sequence is first segmented into two regions, namely, foreground and background. These regions are then encoded using the same coder but with different rate control and quantization level. In this way, the image quality of the more important foreground region can be improved by encoding it with more bits at the expense of background image quality. In this paper, we present a summary of the face segmentation technique required by this approach and then describe the implementation of this foreground/background (FB) coding scheme on the H.261 framework. The discriminatory quantization process and a new rate control strategy for the FB regions are also discussed. In addition, the performance of the FB coding scheme was evaluated by some experimental result carried out on the typical videophone sequences.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Automatic extraction of facial feature points is one of the main problems for semantic coding of videophone sequences at very low bit rates In this contribution, an approach for estimation of the eye and mouth corner point positions is presented. For this proposition, the location informations of the face model are exploited to define search areas for estimation of the eye and mouth corner point positions. Then, the eye and mouth corner point positions are estimated based on a template matching technique with eye and mouth corner templates. Finally, in order to verify these estimated corner point positions, some geometric conditions between the corner point positions and the center point positions of the eyes and the mouth are exploited. The proposed algorithm has been applied to test sequences Claire and Miss America with a spatial resolution corresponding to CIF and a frame rate of 10 Hz.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The paper describes a coding optimization strategy in conformity with the recent ITU-T H.263 Recommendation for videophone sequences at bitrates less than 64 Kbit/s. The optimization algorithm jointly selects the temporal position of the frames to be coded and the coding mode (l, P or PB) of the selected frames. The decision is based on the observation, on a group of frames, of an 'activity' parameter, representing the variation of each frame with respect to the last coded one. The proposed strategy produces coded sequences with average frame rates lower than those produced by a non optimized coder, and a better visual quality of the single frame. However, the activity parameters evaluation, and the observation of several candidates, requires a greater delay, buffer size and complexity of the coding algorithm.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We report on the design, design issues, fabrication and performance of a log-polar CMOS image sensor. The sensor is developed for the use in a videophone system for deaf and hearing impaired people, who are not capable of communicating through a 'normal' telephone. The system allows 15 detailed images per second to be transmitted over existing telephone lines. This framerate is sufficient for conversations by means of sign language or lip reading. The pixel array of the sensor consists of 76 concentric circles with (up to) 128 pixels per circle, in total 8013 pixels. The interior pixels have a pitch of 14 micrometers, up to 250 micrometers at the border. The 8013-pixels image is mapped (log-polar transformation) in a X-Y addressable 76 by 128 array.

videoconferencing applications. The algorithm analyzes the image from the perspective of the human viewer, classifiespixels according to local characteristics, and applies combinations of linear and non-linear filters to each class. Edge pixelsare sharpened, pixels near edges have a median filter applied to mitigate mosquito noise, and pixels in monotonous regionsare smoothed to dampen blocking. The algorithm requires limited processing and can be used independently of thevideoconferencing standard that is used. Objective analysis shows that errors are decreased for a majority of pixels. Insubjective tests using low bit-rate coding standards, processed sequences were judged to significantly improve the imagequality of the decoded sequences in all cases.Keywords: DSP, videoconferencing, postprocessing, low bit-rates, artifacts, blocking, mosquito noise1. INTRODUCTIONThe encoding/decoding process of video teleconferencing and videophone applications introduces unwanted noiseartifacts into the reconstructed sequences. These artifacts detract from the intelligibility of the transmission and areespecially evident when using very low bit-rates. Postprocessing routines can be effective tools in eliminating or at leastmasking much of the blocking and mosquito noise found in decoded images.Minami and Zakhor [1] demonstrated that blocking artifacts could be reduced using an optimization technique on DCTcoefficients when the block boundaries are clearly defined. This optimization scheme significantly reduces blocking, butrelies on a full set of DCT coefficients for the entire image. Video codec standards that employ motion prediction, such as

Recently, MPEG-4 is being formed to study very-low-bit-rate (VLBR) video coding for applications in videotelephony. In this paper, we propose a possible postprocessing technique for VLBR coding. In videophone applications, temporal subsampling is a simple technique which can be combined with other compression schemes to achieve very large compression ratio, so as to satisfy the VLBR requirement. As a result, object motions tend to be jerky and disturbing to the human eyes. To smooth out object motions, we propose a postprocessing technique, motion compensated temporal interpolation (MCTI), to increase the instantaneous decoder frame rate. In MCTI, block-based exhaustive motion search is used to establish temporal association between two reconstructed frames. Both forward and backward searches are used to account for uncovered and newly covered areas properly. With MCTI, we show that one or more frames can be interpolated with acceptable visual quality. After showing the feasibility of MCTI, we propose a fast algorithm FMCTI with reduced computation requirement and negligible performance degradation.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

For applications such as videophone or videoconferencing, we all agree that foreground object is more important than the background information. Nevertheless, in most conventional video codecs, such as H.261, due to the variation of the background lighting, noise or other unpredictable reasons, a large amount of available bandwidth is used to code the residue blocks at the background. Thus, it makes the coding inefficient when we consider the subjective visual quality of the foreground object. In this paper we proposed an object-based coding method. It tries to identify the foreground object and then put most of the resources to code it. We suggest a block-based DWT as a coding kernel because of its simple structure and outstanding performance. Experimental results show that we can compress a video sequence down to below 20 kb/s while having a reasonable visual quality.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Digitalized video and audio system has become the trend of the progress in multimedia, because it provides great performance in quality and feasibility of processing. However, as the huge amount of information is needed while the bandwidth is limitted, data compression plays an important role in the system. Say, for a 176 x 144 monochrornic sequence with 10 frames/sec frame rate, the bandwidth is about 2Mbps. This wastes much channel resource and limits the applications. MPEG (moving piciure ezperi group) standardizes the video codec scheme, and it performs high compression ratio while providing good quality. MPEG-i is used for the frame size about 352 x 240 and 30 frames per second, and MPEG-2 provides scalibility and can be applied on scenes with higher definition, say HDTV (high definition ielevision). On the other hand, some applications concerns the very low bitrate, such as videophone and video-conferencing. Because the channel bandwidth is much limitted in telephone network, a very high compression ratio must be required. For the channel bandwidth as low as 28.8 Kbps (V.42), full-duplex is necessary, and 4Kbps is reserved for one-way speech, the effective bandwidth available is only about 10Kbps. As a result, the digital video signal should be compressed 200 times. For conventional codec scheme, such as 11.261, the performance is poor when the bit-rate is about 64Kbps. Therefore, MPEG-4 is being developed to satisfy the demand. In order to encode video signal, MC (motion compensated) method was widely used in standard systems, in which fix-block based algorithm was applied. However, there are some disadvantages such as block-effect that degrades the performance. To fit these applications, there are many approaches. For instance, model-based coding2 and analysis-synthesis coding.4 Model based coding is a very popular topic of survey in very low bit rate system. Though the encoded pictures are much constrained only in slow-moving of human face, the transmitted information is very little because only some parameters of motion of models must be sent. However, the technique is still at beginning because how to analyze and extract the parameters in a moving sequence is very difficult and complex. Besides, the moving sequence is much limitted in specific patterns. That is, if an unexpected pattern existed in the picture, say, a raising hand, the system may cost even much more both in complexity and bit-rate. And as a commercial consideration, most customers may not accept an "synthesized" countenance of his friend or relative in the screen. The other branch is "object-oriented analysis-synthesis coding" . In the approach, the contents in the picture is classified into background, model compliance object, and model failure object. The model-compliance object is coded by its motion and shape; while the model-failure object is coded by its color (including luminance and chrominance) and shape. The coding of background is unnecessary. Head and shoulder usually belong to the model-compliance objects and the details in face such as eyes and mouth are model-failure objects. This method immunes from the variety of picture patterns compared with the model-based approach, however, the performance is not good enough to fit the practical demand. Besides, the complex analysis is also a major burden. There are many researches about this topic recently and lots of modifications and adaptive schemes were proposed. As mentioned above, object-oriented coding is block-effect free. Besides, it is an efficient approach for coding the scene in video-conferencing application because only simple head and shoulder occupies the major part. In the view of image analysis, object-oriented approach contains much direct information than conventional waveform coding thus it can satisfy the requirements. In this paper, a pseudo object-oriented coding system is proposed. It is based on a pel-wised motion estimation and arbitrarily shaped transform. Instead of block matching scheme, the motion estimation is realized by a modified optical flow algorithm (MOFA). Because ofthe quasi-homogeneous property of motion field, the objects can be extracted by simpler segmentation. These objects are applied with arbitrarily shaped transform (AST). Briefly speaking, MOFA reduces temporal redundancy while AST reduces spatial redundancy. Since AST is applied on motion field rather than color field, it is quite different from conventional coding system.

The aim of this paper is to investigate the different components of a laser scanner for industrial applications like quality control or future domestic applications like the front end of a videophone system. The scanner implemented in this paper is based in the laser range finding principle.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper an innovative approach to videophone coding is described, based on a 3D model of a human face with a complex muscle structure capable to reproduce faithfully the basic facial expressions and mimics. The muscle structure is organized in a set of interconnected fibers covering the whole surface of the face and characterized by predefined mechanical properties. Muscles can be activated through the direct stimulation of each individual fiber or, indirectly, by interaction with adjacent stimulated fibers. Through the analysis algorithms performed at the transmitter, the input video sequence is processed to extract a set of suitable facial control features and to estimate the muscle parameters. These parameters, namely the value of the estimated muscle stimula, are then quantized, coded and transmitted to the receiver where they are applied to the model to synthesize the corresponding facial expression. In the paper the muscle structure is described together with the coding/decoding algorithms which have been implemented on a multiprocessor AT&T Pixel Machine. Some samples of the achieved experimental results are also presented to show the significant subjective quality of the reconstructed sequence.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We present some preliminary results of our work on model-based compression of video sequences of a person’s face, in the context of teleconferencing and videophone applications. The emphasis is on the difficult and challenging problem of analysis. Algorithms are presented for extracting and tracking key feature points on the face, and for estimating the global and local motion of the head/face.

A layered Pyramid image coding scheme suitable for interworking of videoconferencing and videophone services is presented in this paper. The scheme proposed takes advantage of inter- layer embedded motion prediction to compress image data on the upper Pyramid enhancement layer. As an important advantage of this scheme we have preserved independent of the coding in the two layers using embedded motion compensation. Thus no information about the coding procedure on the lower resolution layer is needed to encode the upper layer enhancement information. The results are encouraging, showing that the scheme reconstructs images of both CIF and QCIF resolution with good quality and that the scheme is more data efficient than a comparable SimulCast approach.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A simple but efficient video codec is built for low-bit-rate videophone applications. The goal of this project is to construct a video codec at a low cost and good performance. In order to reduce hardware complexity, a simple interframe predictive coding structure is selected. The input pictures are partitioned into 2-D blocks, and only the blocks with significant frame differences are coded and transmitted to the decoder. The Block-list-transform (BLT) coding algorithm, a generalized DPCM, is used to encode the frame differences between the previously reconstructed picture and the current picture. Preliminary results using this coding system at 64 kbits/sec show reasonable coding performance on typical videophone sequences.

Interframe coding schemes though sensitive to the propagative affects of channel-errors, are much more efficient than intraframe coding schemes, when using similar data-compression techniques. Moreover, since in interframe coding, the instantaneous data-rate may vary drastically, it can benefit from the bits-on-demand property of packet-networks. We propose a layered coding-structure that contains a self-contained basic inter-frame codec as the core, and supplementary additive extended-codecs. The inter-frame codec uses a slightly modified version of motion-compensated DCT coding scheme proposed by the CCITT in reference model 8 for pX64kbit/ s videophone. It encodes the lower spatial resolution image sequence, with an adequate temporal resolution. The encoded-data it produces is assigned the highest priority. The extended-codecs allow enhancement of temporal and spatial resolutions, at the cost of increase in the data-rate. They encode the interpolative differences between the upsampled local decoded output of the basic-codec and the higher resolution input. The encoded-data of the extended codecs is assigned lower priority.