Search

With the development of the broadband network, video communications such as videophone, video distribution, and IPTV services are beginning to become common. In order to provide these services appropriately, we must manage them based on subjective video quality, in addition to designing a network system based on it. Currently, subjective quality assessment is the main method used to quantify video quality. However, it is time-consuming and expensive. Therefore, we need an objective quality assessment technology that can estimate video quality from video characteristics effectively. Video degradation can be categorized into two types: spatial and temporal. Objective quality assessment methods for spatial degradation have been studied extensively, but methods for temporal degradation have hardly been examined even though it occurs frequently due to network degradation and has a large impact on subjective quality. In this paper, we propose an objective quality assessment method for temporal degradation. Our approach is to aggregate multiple freeze distortions into an equivalent freeze distortion and then derive the objective video quality from the equivalent freeze distortion. Specifically, our method considers the total length of all freeze distortions in a video sequence as the length of the equivalent single freeze distortion. In addition, we propose a method using the perceptual characteristics of short freeze distortions. We verified that our method can estimate the objective video quality well within the deviation of subjective video quality.

Architecture of an IP-based narrow-band videophone system is proposed in this paper for convenient videophone calls between any two computers even if being placed in two different LANs within network agents. The bandwidth need of each call is less than 256 kbps. The system consists of two kinds of entities: Videophone Terminals (VPT) and a Video Call Server (VCS). A VPT is actually a microcomputer program, composed of 4 primary parts, an audio codec, a video codec, a media deliverer/receiver and a call controller. The basic functions of the VCS include videophone number generation and management, access admission and address resolution. The VCS with a public IP address plays an important role in the system especially when a video call has to penetrate through network agents. Each VPT in the system gets its own external transport address from the VCS through registration process. A calling VPT would receive the external transport address of the called VPT from the VCS through address resolution. The proposed system works and is helpful to accelerate the realization of people's videophone dream over IP networks.

In low bit-rate or very low bit-rate video applications, such as videophone and video conferencing, it is difficult for traditional methods to gain a satisfactory image quality. They often suffer from some shortcomings: images with artifacts caused by limited bandwidth and motion discontinuity caused by frame skipping when bit-streams are transmitted through a fixed rate channel in low-delay applications. To attack the two problems above, we propose a content-based rate control method for videophone and video conferencing applications. Our work is focused on three aspects: analysis of bit-streams in low bit-rate video applications, content-based bit allocation to facial areas and background, and low-delay rate control in object-layer. Experiments show that the proposed method can achieve better subjective quality compared with that of the test model TMN5 of H.263, and slightly worse performance in PSNR, but with significantly better subjective quality in the facial areas, compared with that of the TMN8 of H.263+.

The transmission of real-time images and video over wireless communication channels is still a challenge problem. Digital compressed images are sensitive to bit errors which are typical in wireless communications. Moreover, the bandwidth at the air interface is currently a limiting factor because the first and second generation of mobile phone standards mainly support voice communications. In this paper, we present our study of real-time image traffic over a radio link -- the aim of this research is for videophone applications. In this study, we use the Discrete Wavelet Transform (DWT) to compress images and a Code Division Multiple Access (CDMA) link to transfer images over wireless communication channels. The results of the experiment show that it is possible to transfer 4 QCIF images per second over a CD MA link with minor degradation in image quality. This study was investigated by the VLSI Signal, Image and Video Processing Research Laboratory at the University of California, San Diego (UCSD).© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

One of the largest sources of compression in vision is the sensitivity and resolution reduction as a function of eccentricity. However, utilization of this visual property has been limited to system that directly measure the viewer's gaze position. We have applied visual eccentricity models to videophone compression applications without using eye tracking by combining the visual model with a face tracking algorithm. In lieu of a gaze-detector, we assume the gaze will be directed to the faces appearing in images. The incorporation of both resolution as well as sensitivity- based eccentricity models in a low bitrate video compression standard will be discussed. For videophone applications, the reduction in bitrate while retaining similar image quality is up to 50 percent. Problems arising from the improved temporal sensitivity of the periphery, despite its reduced spatial bandwidth, will be discussed.

A VideoBeamTM portable laser communicator has been developed which provides full duplex communication links consisting of high quality analog video and stereo audio. The 3.2-pound unit resembles a binocular-type form factor and has an operational range of over two miles (clear air) with excellent jam-resistance and low probability of interception characteristics. The VideoBeamTM unit is ideally suited for numerous military scenarios, surveillance/espionage, industrial precious mineral exploration, and campus video teleconferencing applications.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper presents a low bit rate videophone system for deaf people communicating by means of sign language. Classic video conferencing systems have focused on head and shoulders sequences which are not well-suited for sign language video transmission since hearing impaired people also use their hands and arms to communicate. To address the above-mentioned functionality, we have developed a two-step content-based video coding system based on: (1) A segmentation step. Four or five video objects (VO) are extracted using a cooperative approach between color-based and morphological segmentation. (2) VO coding are achieved by using a standardized MPEG-4 video toolbox. Results of encoded sign language video sequences, presented for three target bit rates (32 kbits/s, 48 kbits/s and 64 kbits/s), demonstrate the efficiency of the approach presented in this paper.

This paper presents a layered video-coding scheme [1] for very low bit rate videophone. The lowertwo layers (the first and second layer) of this scheme are compatible with ITU-T H.263 [2] in order to becompatible with other schemes. The first layer (basic layer) is normal H.263 withouta priori knowledgeof the image content, which is robust to image content. The second layer introduces the typical head-shoulder assumption, and provides subjectively improved quality of reconstructed image. The third layeris based on the same model as layer two, but further improves the subjective quality by employing theshape information of the interested objects. Manipulation functionality is supported in the third layer.When the typical head-shoulder hypothesis is true, the coding scheme can upgrade its coding layer toobtain better subjective quality. On the other hand, the coding scheme can withdraw to robust lowerlayers when the a priori knowledge is not available. In this paper, a novel motion estimation criterion isintroduced which can be integrated into most of present strategies to improve the overall performance. AB-spline approach is introduced in this paper to encode the object contour instead of chain code.Keywords: layered-coding, object-based, contour-coding, arbitrary-shaped, motion estimation criteria

A suite of bandwidth efficient image codecs are presented for the use in second-generation wireless systems, such as the American IS-54 and IS-95 systems, the Pan-European GSM system and the Japanese digital cellular system. The proposed codecs are configured to operate at 9.6K bits per second and are suitable for Quarter Common Intermediate Format videophone sequences, scanned at 10 frames per second. The new image codecs employ the orthonormal wavelet transform to decompose the displaced frame difference data for each frame, into four frequency subbands. The wavelet coefficients within the frequency subbands are then encoded using vector quantization. Comparison measures are undertaken for the two-stage pairwise nearest neighbor (PNN) algorithm and the designs are rated upon their ability to coherently reconstruct an efficient codebook from a training sequence of vector coefficients. It was found that the two- stage PNN algorithm constitutes a valuable compromise in terms of computational complexity with only negligible performance loss. When the codecs were configured to operate at 9.6Kpbs, the average peak signal to noise ratio of the two stage PNN and the adaptive algorithms were in excess of 28kB and 30dB respectively.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This research addresses the use of face segmentation to improve the subjective quality of videophone sequence encoded by the H.261-compliant coder. In this approach, each frame of the input video sequence is first segmented into two regions, namely, foreground and background. These regions are then encoded using the same coder but with different rate control and quantization level. In this way, the image quality of the more important foreground region can be improved by encoding it with more bits at the expense of background image quality. In this paper, we present a summary of the face segmentation technique required by this approach and then describe the implementation of this foreground/background (FB) coding scheme on the H.261 framework. The discriminatory quantization process and a new rate control strategy for the FB regions are also discussed. In addition, the performance of the FB coding scheme was evaluated by some experimental result carried out on the typical videophone sequences.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A rate-distortion framework is used to define a displacement vector-field estimation technique for use in video coding. This technique achieves maximum reconstructed image quality under the constraint of a target bitrate for the coding of the vector sequence. Use of this technique is evaluated for two application areas in which the need for high compression of displacement vector fields in particularly acute. The first is motion-field coding for very low bit rate image sequence transmission as in videophone applications. The second application area is coding for the transmission of disparity fields. This is needed for the generation at the receiver of intermediate viewpoints through spatial interpolation. It is also needed in a number of other applications requiring accurate depth knowledge, including 3D medical data transmission and transmission of scenes to be postprocessed using depth-keyed segmentation. Experimental results illustrating the performance of the proposed technique in these application areas are presented and evaluated.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Digitalized video and audio system has become the trend of the progress in multimedia, because it provides great performance in quality and feasibility of processing. However, as the huge amount of information is needed while the bandwidth is limitted, data compression plays an important role in the system. Say, for a 176 x 144 monochromic sequence with 10 frames/sec frame rate, the bandwidth is about 2Mbps. This wastes much channel resource and limits the applications. MPEG (moving picttre ezpert groip) standardizes the video codec scheme, and it performs high compression ratio while providing good quality. MPEG-i is used for the frame size about 352 x 240 and 30 frames per second, and MPEG-2 provides scalibility and can be applied on scenes with higher definition, say HDTV (high definition television). On the other hand, some applications concerns the very low bit-rate, such as videophone and video-conferencing. Because the channel bandwidth is much limitted in telephone network, a very high compression ratio must be required. ITU-T announced the H.263 video coding standards to meet the above requirements.8 According to the simulation results of TMN-5,22 it outperforms 11.263 with little overhead of complexity. Since wireless communication is the trend in the near future, low power design of the video codec is an important issue for portable visual telephone. Motion estimation is the most computation consuming parts in the whole video codec. About 60% of the computation is spent on this parts for the encoder. Several architectures were proposed for efficient processing of block matching algorithms. In this paper, in order to meet the requirements of 11.263 and the expectation of low power consumption, a modified sandwich architecture in21 is proposed. Based on the parallel processing philosophy, low power is expected and the generation of either one motion vector or four motion vectors with half-pixel accuracy is achieved concurrently. In addition, we will present our solution how to solve the other addition modes in 11.263 with the proposed architecture.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Automatic extraction of facial feature points is one of the main problems for semantic coding of videophone sequences at very low bit rates In this contribution, an approach for estimation of the eye and mouth corner point positions is presented. For this proposition, the location informations of the face model are exploited to define search areas for estimation of the eye and mouth corner point positions. Then, the eye and mouth corner point positions are estimated based on a template matching technique with eye and mouth corner templates. Finally, in order to verify these estimated corner point positions, some geometric conditions between the corner point positions and the center point positions of the eyes and the mouth are exploited. The proposed algorithm has been applied to test sequences Claire and Miss America with a spatial resolution corresponding to CIF and a frame rate of 10 Hz.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The paper describes a coding optimization strategy in conformity with the recent ITU-T H.263 Recommendation for videophone sequences at bitrates less than 64 Kbit/s. The optimization algorithm jointly selects the temporal position of the frames to be coded and the coding mode (l, P or PB) of the selected frames. The decision is based on the observation, on a group of frames, of an 'activity' parameter, representing the variation of each frame with respect to the last coded one. The proposed strategy produces coded sequences with average frame rates lower than those produced by a non optimized coder, and a better visual quality of the single frame. However, the activity parameters evaluation, and the observation of several candidates, requires a greater delay, buffer size and complexity of the coding algorithm.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We report on the design, design issues, fabrication and performance of a log-polar CMOS image sensor. The sensor is developed for the use in a videophone system for deaf and hearing impaired people, who are not capable of communicating through a 'normal' telephone. The system allows 15 detailed images per second to be transmitted over existing telephone lines. This framerate is sufficient for conversations by means of sign language or lip reading. The pixel array of the sensor consists of 76 concentric circles with (up to) 128 pixels per circle, in total 8013 pixels. The interior pixels have a pitch of 14 micrometers, up to 250 micrometers at the border. The 8013-pixels image is mapped (log-polar transformation) in a X-Y addressable 76 by 128 array.

videoconferencing applications. The algorithm analyzes the image from the perspective of the human viewer, classifiespixels according to local characteristics, and applies combinations of linear and non-linear filters to each class. Edge pixelsare sharpened, pixels near edges have a median filter applied to mitigate mosquito noise, and pixels in monotonous regionsare smoothed to dampen blocking. The algorithm requires limited processing and can be used independently of thevideoconferencing standard that is used. Objective analysis shows that errors are decreased for a majority of pixels. Insubjective tests using low bit-rate coding standards, processed sequences were judged to significantly improve the imagequality of the decoded sequences in all cases.Keywords: DSP, videoconferencing, postprocessing, low bit-rates, artifacts, blocking, mosquito noise1. INTRODUCTIONThe encoding/decoding process of video teleconferencing and videophone applications introduces unwanted noiseartifacts into the reconstructed sequences. These artifacts detract from the intelligibility of the transmission and areespecially evident when using very low bit-rates. Postprocessing routines can be effective tools in eliminating or at leastmasking much of the blocking and mosquito noise found in decoded images.Minami and Zakhor [1] demonstrated that blocking artifacts could be reduced using an optimization technique on DCTcoefficients when the block boundaries are clearly defined. This optimization scheme significantly reduces blocking, butrelies on a full set of DCT coefficients for the entire image. Video codec standards that employ motion prediction, such as

Recently, MPEG-4 is being formed to study very-low-bit-rate (VLBR) video coding for applications in videotelephony. In this paper, we propose a possible postprocessing technique for VLBR coding. In videophone applications, temporal subsampling is a simple technique which can be combined with other compression schemes to achieve very large compression ratio, so as to satisfy the VLBR requirement. As a result, object motions tend to be jerky and disturbing to the human eyes. To smooth out object motions, we propose a postprocessing technique, motion compensated temporal interpolation (MCTI), to increase the instantaneous decoder frame rate. In MCTI, block-based exhaustive motion search is used to establish temporal association between two reconstructed frames. Both forward and backward searches are used to account for uncovered and newly covered areas properly. With MCTI, we show that one or more frames can be interpolated with acceptable visual quality. After showing the feasibility of MCTI, we propose a fast algorithm FMCTI with reduced computation requirement and negligible performance degradation.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

For applications such as videophone or videoconferencing, we all agree that foreground object is more important than the background information. Nevertheless, in most conventional video codecs, such as H.261, due to the variation of the background lighting, noise or other unpredictable reasons, a large amount of available bandwidth is used to code the residue blocks at the background. Thus, it makes the coding inefficient when we consider the subjective visual quality of the foreground object. In this paper we proposed an object-based coding method. It tries to identify the foreground object and then put most of the resources to code it. We suggest a block-based DWT as a coding kernel because of its simple structure and outstanding performance. Experimental results show that we can compress a video sequence down to below 20 kb/s while having a reasonable visual quality.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Digitalized video and audio system has become the trend of the progress in multimedia, because it provides great performance in quality and feasibility of processing. However, as the huge amount of information is needed while the bandwidth is limitted, data compression plays an important role in the system. Say, for a 176 x 144 monochrornic sequence with 10 frames/sec frame rate, the bandwidth is about 2Mbps. This wastes much channel resource and limits the applications. MPEG (moving piciure ezperi group) standardizes the video codec scheme, and it performs high compression ratio while providing good quality. MPEG-i is used for the frame size about 352 x 240 and 30 frames per second, and MPEG-2 provides scalibility and can be applied on scenes with higher definition, say HDTV (high definition ielevision). On the other hand, some applications concerns the very low bitrate, such as videophone and video-conferencing. Because the channel bandwidth is much limitted in telephone network, a very high compression ratio must be required. For the channel bandwidth as low as 28.8 Kbps (V.42), full-duplex is necessary, and 4Kbps is reserved for one-way speech, the effective bandwidth available is only about 10Kbps. As a result, the digital video signal should be compressed 200 times. For conventional codec scheme, such as 11.261, the performance is poor when the bit-rate is about 64Kbps. Therefore, MPEG-4 is being developed to satisfy the demand. In order to encode video signal, MC (motion compensated) method was widely used in standard systems, in which fix-block based algorithm was applied. However, there are some disadvantages such as block-effect that degrades the performance. To fit these applications, there are many approaches. For instance, model-based coding2 and analysis-synthesis coding.4 Model based coding is a very popular topic of survey in very low bit rate system. Though the encoded pictures are much constrained only in slow-moving of human face, the transmitted information is very little because only some parameters of motion of models must be sent. However, the technique is still at beginning because how to analyze and extract the parameters in a moving sequence is very difficult and complex. Besides, the moving sequence is much limitted in specific patterns. That is, if an unexpected pattern existed in the picture, say, a raising hand, the system may cost even much more both in complexity and bit-rate. And as a commercial consideration, most customers may not accept an "synthesized" countenance of his friend or relative in the screen. The other branch is "object-oriented analysis-synthesis coding" . In the approach, the contents in the picture is classified into background, model compliance object, and model failure object. The model-compliance object is coded by its motion and shape; while the model-failure object is coded by its color (including luminance and chrominance) and shape. The coding of background is unnecessary. Head and shoulder usually belong to the model-compliance objects and the details in face such as eyes and mouth are model-failure objects. This method immunes from the variety of picture patterns compared with the model-based approach, however, the performance is not good enough to fit the practical demand. Besides, the complex analysis is also a major burden. There are many researches about this topic recently and lots of modifications and adaptive schemes were proposed. As mentioned above, object-oriented coding is block-effect free. Besides, it is an efficient approach for coding the scene in video-conferencing application because only simple head and shoulder occupies the major part. In the view of image analysis, object-oriented approach contains much direct information than conventional waveform coding thus it can satisfy the requirements. In this paper, a pseudo object-oriented coding system is proposed. It is based on a pel-wised motion estimation and arbitrarily shaped transform. Instead of block matching scheme, the motion estimation is realized by a modified optical flow algorithm (MOFA). Because ofthe quasi-homogeneous property of motion field, the objects can be extracted by simpler segmentation. These objects are applied with arbitrarily shaped transform (AST). Briefly speaking, MOFA reduces temporal redundancy while AST reduces spatial redundancy. Since AST is applied on motion field rather than color field, it is quite different from conventional coding system.

The paper investigates the capabilities of a novel 2D motion estimation and object-background segmentation algorithm to be used for low bit rate videophone compression. The algorithm is based on a model of the image inspired by the work of Geman and Geman [GG84]. Observable are three frames, the previous, the current, and the successive ones, and the edges of the current frame. The algorithm tries to minimize a global cost function, depending on local displacement and segmentation values, whose minimum corresponds to the desired displacement and segmentation fields. The performances of the algorithm have been measured on the total residual error and on the ratio of the final to initial costs. The algorithm has been tested on artificial and natural image sequences and the preliminary experimental results have been satisfactory, confirming the validity of the model and showing a perceivable improvement over block matching. Moreover the algorithm is suitable for distributed parallel processing.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The mismatch between dynamic ranges of intensities in non—uniformly illuminated videophonescenes and of intensities reproducable at display devices is addressed. Displayed portrait images with no details visible in some facial regions result from this mismatch and lead in turn to a reduced impression of telepresence.A novel image processing approach for dynamic range compression of videophone portrait imagesis presented in this paper. Unlike conventional methods, it provides compressed images with a mostnatural appearance. The basic idea is to simulate a more uniform scene illumination in the video signalsof videophone portrait images. This requires a determination of the scene's reflectance function. Anextension of the classical lightness approach of Land and McCann [5] to curved surfaces is proposedfor this task.Keywords: dynamic range compression, videotelephony, lightness algorithm 1 INTRODUCTION In the near future visual telecommunication will become common practice via videophone sets fromhomes, offices etc. Like with today's plain old telephones, users will want these terminals to work intheir existing environments. They would not accept specially designed videophone studios. However, inordinary domestic or working environments one is faced with various lighting conditions (bright sunlight,dark daylight, or artificial light) [8]. Specifically, if direct sunlight is falling in, the illumination of thescenes can be highly non—uniform. In these cases, intensity variations between those portions of a faceturned towards and those turned away from the sun may easily exceed ratios of 500 : 1. Electronicallyreproduced images of such scenes show obscured regions because of the limited dynamic range of commondisplay devices1. Figure 1 illustrates this for a typical videophone portrait image.Using conventional dynamic range compression techniques such as histogram transformation [3] orhomomorphic filtering [9], details may become visible in otherwise obscured regions of the image. But,these techniques generate compressed images which tend to appear unrealistic2 {2]. Thus, they are unsuit-able to resolve dynamic range problems in videotelephony or videoconferencing, since these services areaimed at giving the impression of a communication situation similar to that of real face—to—face meetings

The implicit and explicit use of 3D models for image sequence coding is discussed. For implicit use, a 3D model can be incorporated into motion compensating prediction. A scheme that estimates the displacement vector field with a rigid body motion constraint by recovering epipolar lines from an unconstrained displacement estimate and then repeating block matching along the epipolar line is proposed. Experimental results show that an improved displacement vector field can be obtained with a rigid body motion constraint. As an example for explicit use, various results with a facial animation model for videotelephony are discussed. A 13 X 16 B-spline mask can be adapted automatically to individual faces and is used to generate facial expressions based on FACS. A depth-from-defocus range camera suitable for real-time facial motion tracking is described. Finally, the real-time facial animation system `Traugott' is presented that has been used to generate several hours of broadcast video. Experiments suggest that a videophone system based on facial animation might require a transmission bitrate of 1 kbit/s or below.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper describes a `very low bitrate visual telephony application' demonstrator which was designed to be used on the Public Switched Telephony Network for many multimedia purposes. This development was done by CNET in coordination with the european COST211ter project with the aim to demonstrate videotelephony at very low bit rates. The main concern was to optimize a video coding algorithm based on the CCITT H.2161 existing standard and directly derived from the COST211ter simulation model. The different signals which are needed for a videotelephony communication, video, speech, data and control are modulated and transmitted at a bitrate contained between 9.6 kbit/s and 28.8 kbit/s. The description of the demonstrator is given, including video algorithm and system multiplex specifications. The reasons of the choice of the video format and algorithm are also discussed. A friendly software application has been developed to run videotelephony within a Macintosh computer environment. This program uses the QuickTime routines to record and to play the videophone pictures to or from the hard disk. Single pictures or large sequences can be grabbed to the hard disk. Data can also be transmitted by opening, through the audio/video multiplex a data channel of some kbit/s in the video channel, allowing minimal groupwave application.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The goal of the ISO project, designated MPEG4, is to develop a generic video coding syntax suitable for a wide range of applications such as videophone via the PSTN and mobile radio, security systems, mobile experts, emergency monitoring, educational networks, and networked games. It is anticipated that the coding algorithm will be a significant advancement relative to the basic interframe predictive 8 X 8 DCT design which is used in most digital TV standards today. Examples of advanced coding techniques being considered include fractals, analysis/synthesis, knowledge-based, and semantic coding.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The aim of this paper is to investigate the different components of a laser scanner for industrial applications like quality control or future domestic applications like the front end of a videophone system. The scanner implemented in this paper is based in the laser range finding principle.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper an innovative approach to videophone coding is described, based on a 3D model of a human face with a complex muscle structure capable to reproduce faithfully the basic facial expressions and mimics. The muscle structure is organized in a set of interconnected fibers covering the whole surface of the face and characterized by predefined mechanical properties. Muscles can be activated through the direct stimulation of each individual fiber or, indirectly, by interaction with adjacent stimulated fibers. Through the analysis algorithms performed at the transmitter, the input video sequence is processed to extract a set of suitable facial control features and to estimate the muscle parameters. These parameters, namely the value of the estimated muscle stimula, are then quantized, coded and transmitted to the receiver where they are applied to the model to synthesize the corresponding facial expression. In the paper the muscle structure is described together with the coding/decoding algorithms which have been implemented on a multiprocessor AT&T Pixel Machine. Some samples of the achieved experimental results are also presented to show the significant subjective quality of the reconstructed sequence.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We present some preliminary results of our work on model-based compression of video sequences of a person’s face, in the context of teleconferencing and videophone applications. The emphasis is on the difficult and challenging problem of analysis. Algorithms are presented for extracting and tracking key feature points on the face, and for estimating the global and local motion of the head/face.

A layered Pyramid image coding scheme suitable for interworking of videoconferencing and videophone services is presented in this paper. The scheme proposed takes advantage of inter- layer embedded motion prediction to compress image data on the upper Pyramid enhancement layer. As an important advantage of this scheme we have preserved independent of the coding in the two layers using embedded motion compensation. Thus no information about the coding procedure on the lower resolution layer is needed to encode the upper layer enhancement information. The results are encouraging, showing that the scheme reconstructs images of both CIF and QCIF resolution with good quality and that the scheme is more data efficient than a comparable SimulCast approach.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The Broadband Integrated Services Digital Network (BISDN) based on lightwave technology is supposed to become the all-purpose exchange area communications network of the future. All digital video services are integrated with applications ranging from videophone teleconferencing to digital TV (signals according to the CCIR rec. 601) and High Definition TV (HDTV) distribution. A desirable feature of the various video services is the upward and downward compatibility in resolution in order to guarantee a free exchange of services transmitters and receivers. This paper proposes an n-level progressive hierachical intraframe coding scheme based on subband coding. In this scheme several spatial low-resolution services are available as subsets of the coded HDTV data which can directly be received at lower bit rates. Progressive coding of the HDTV signal is employed in order to prevent quantization errors from propagating to higher resolution signals. Special attention is given to the design of the quantizers required for the progressive coding and to the incorporation of the side panels coding.© (1990) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A coder concept for coding of digital television sequences at 1 Mbit/s is presented. Emerging from the standard hybrid codec for videophone at 64 kbit/s an advanced concept has been investigated which considers the special properties of real world image sequences like unrestricted motion, zoom/pan, changes of illumination and scene cuts. These effects, which cannot be considered for videophone applications because of the extremly low available data rate, make it necessary to extend the standard hybrid coding scheme for 64 kbit/s. To reduce the amount of data the original image sequence in CCIR Rec. 601 format is spatially and temporarily subsampled. The resulting sequence is in accordance with the Common Intermediate Format (CIF) but consists of 25 noninterlaced frames/second. Motion estimation and compensation are improved by applying an algorithm for illumination correction. Additionally zoom and pan, which is widely used in common TV-scenes, is compensated by processing the motion vector field of the previous motion estimation. Scene cuts are detected and considered by intraframe coding of the next frame. Besides every 10th frame is intraframe coded to allow fast forward and reverse searching start at any arbitrary point within the sequence.

The authors review data from cost analyses of fiber loop networks. It is argued that fiber in the local loop will provide cost-effective transport of services to about the T1 rate, making videophone and similar video services attractive and cost-effective in the near term. In contrast, it is suggested that fiber networks will not provide cost competition with coaxial cable TV networks for five to ten years.

A simple but efficient video codec is built for low-bit-rate videophone applications. The goal of this project is to construct a video codec at a low cost and good performance. In order to reduce hardware complexity, a simple interframe predictive coding structure is selected. The input pictures are partitioned into 2-D blocks, and only the blocks with significant frame differences are coded and transmitted to the decoder. The Block-list-transform (BLT) coding algorithm, a generalized DPCM, is used to encode the frame differences between the previously reconstructed picture and the current picture. Preliminary results using this coding system at 64 kbits/sec show reasonable coding performance on typical videophone sequences.

Interframe coding schemes though sensitive to the propagative affects of channel-errors, are much more efficient than intraframe coding schemes, when using similar data-compression techniques. Moreover, since in interframe coding, the instantaneous data-rate may vary drastically, it can benefit from the bits-on-demand property of packet-networks. We propose a layered coding-structure that contains a self-contained basic inter-frame codec as the core, and supplementary additive extended-codecs. The inter-frame codec uses a slightly modified version of motion-compensated DCT coding scheme proposed by the CCITT in reference model 8 for pX64kbit/ s videophone. It encodes the lower spatial resolution image sequence, with an adequate temporal resolution. The encoded-data it produces is assigned the highest priority. The extended-codecs allow enhancement of temporal and spatial resolutions, at the cost of increase in the data-rate. They encode the interpolative differences between the upsampled local decoded output of the basic-codec and the higher resolution input. The encoded-data of the extended codecs is assigned lower priority.

The goal of image coding is to reduce the number of bits required to represent an image or a sequence of images. For the new generation of video coders the coding should be tailored to image understanding and should more and more replace the purely statistical or waveform coding techniques. At present the more conventional algorithms are prefered in the light of a direct hardware realization. Most of these methods are block based and do not consider the global image features. In this paper an attempt is made to extend a hybrid DPCM/transform coding configuration with object based features. For videoconferencing and videophone applications we intend to use a simple method exploiting the knowlegde of typical interpersonal videocommunication scenes such as head and shoulders. The first step is the segmentation of the image into participants and background. For this segmentation information available in the coder is combined with knowledge obtained from the actual analysis. To make the segmentation more robust to various input signals we accept the fact that objects often are more easily recognized in images that have a very low spatial sampling rate. At this low level details are not taken into account which yields a more homogenous segmentation. This leads to a pyramidal image data approach. By applying a coarse to fine structure the procedure starts at a low resolution level and is refined at ever-increasing resolutions. A comparison is given of a coder with and without this a priori knowlegde.