Search

This chapter builds on the foundations laid down in the previous chapter. It describes the human-centered system development framework for intelligent multimedia multiagent systems based on human-centered criteria outlined in the first chapter and the pragmatic considerations and theories discussed in chapter 3, which contribute towards realization of those criteria. The human-centered framework is described in terms of four components, namely, activity-centered analysis, problem solving ontology, transformation agents, and multimedia interpretation, respectively. The three human-centered criteria are used as guidelines for development of the human-centered framework. The pragmatic considerations and contributing theories are used to develop the structure and content, or knowledge base, of the four components. The structure and content are described at the conceptual and computational (transformation agents) level. We start this chapter by describing the external and internal planes of action which underpin the development of the human-centered framework. We follow it with the description of two components human-centered system development framework, namely, activity-centered analysis and problem solving ontology. In the next chapter we continue with the description of the problem solving ontology component and describe the transformation agent and multimedia interpretation component. These four components have been used to define the external and internal planes of human interaction with the environment.

In the last two chapters we have looked into the application of four components of the HCVM. In this chapter we wish to demonstrate two other facets of the human-centered virtual machine, which have not been specifically looked at until now. These are the role of computer-based intermediaries (for back up support in problem solving), distributed communication in human-centered multi-agent systems and using computer-based artifacts to model breakdowns in human decision making. The role of intermediaries and distributed communication is illustrated by way of an intelligent distributed multi-agent multimedia euchre application on the net. In addition, aspects of a sales recruitment and benchmarking application are described to illustrate how computer-based artifact is used to overcome breakdowns in the sales, customer service and telesales recruitment decision making. In fact the modeling of the breakdowns is the prime motivator for the sales managers and sales personnel to use the computerized system for management of recruitment and benchmarking. The sales recruitment system described in this chapter (and developed by the first author) has been used in the industry for approximately the past three years.

In the last three chapters we have described applications of HCVM in medical informatics, image processing, internet games and sales recruitment. In these applications we have looked at multimedia artifacts in terms of how they can be used for improving the representational efficiency, effectiveness and interpretation of computer-based artifacts and also to some extent how they can be used for perceptual problem solving. However, there are interesting research issues and problems associated with management and retrieval of text, image, video, audio artifacts from single and multimedia databases. The area of multimedia information management and retrieval has in recent times become important due to the advent of the internet and World Wide Web (WWW). The solution to a number of issues in this area today lies with traditional solutions used for conventional databases. Recently, intelligent agents and human-centered information retrieval models are being proposed by researchers to address the problems in this area. In order to gain a proper understanding of the various problems and issues involved, we start this chapter by outlining the three evolution stages of this time honored area. We then describe these evolution stages in some detail. We include in this description applications and solutions proposed for multimedia information management and retrieval. We conclude the chapter by outlining open research problems in this area.

Human-centered system development is not a revolutionary concept in computer science and information systems but an evolutionary and enabling one. In this chapter we look at how some areas in computer science and information systems are evolving or moving towards human-centeredness. These areas include intelligent systems, electronic commerce, software engineering, multimedia databases, data mining, enterprise modeling and human-computer interaction. This evolution is based on the need for addressing pragmatic issues in these areas. We follow these pragmatic issues with enabling theories in philosophy, cognitive science, psychology and work-oriented design for human-centered system development framework. These theories are described and discussed in terms of their contributions toward human-centered system development framework. We conclude the chapter with a discussion section that outlines the foundations of the human-centered system development framework described in the chapter.

Electronic commerce (EC) can be broadly seen as the application of information technology and telecommunications to create trading networks where goods and services are sold and purchased, thus increasing the efficiency and the effectiveness of traditional commerce.

The objective of this chapter is to show the application of the conceptual and computational facets of the HCVM in a clinical drug prescription monitoring activity. The application is in the area of infectious diseases (Gorbach et al. 1998; Barrows et al. 1991) and addresses problems related to the gathering of patient symptomatic data, providing diagnostic assistance, and finding inconsistencies in practitioner prescribed treatments compared to those recommended by therapeutic guidelines (TG 1998).

Color is one of the most widely used features for image similarity retrieval. Most of the existing image similarity retrieval schemes employ either global or local color histogramming. In this paper, we explore the use of localized dominant hue and saturation values for color-based image similarity retrieval. This scheme results in a relatively compact representation of color images for similarity retrieval. Experimental results comparing the proposed representation with global and local color histogramming are presented to show the efficacy of the suggested scheme.

Chroma-keying or blue screen matting is an important video editing operational. In blue screen matting, everything in the image that has the 'user specified level for the blue channel' is 'keyed' out and replaced by either another image or a color from a color generator. We develop a technique for blue screen matting that manipulates the image when the information for the image is available only in compressed form such as a JPEG or MPEG bitstream. Specifically, for the compressed domain approach, we show that the matting process is a convolution operation; hence, we develop a DCT convolution theorem. The DCT convolution theorem can be used to show that the compressed domain approach proposed in this paper provides a significant reduction in computation complexity compared to previously developed approaches which were mostly adhoc techniques. The convolution theorem exploits the sparseness as well as the orthogonality of the data available in the DCT domain and thus yields an efficient algorithm for he chroma-keying or blue screen matting. The algorithm extends to the DCT domain concepts such as alpha channel and premultiplied alpha. The method is also extended to MPEG video domain wherein we explore the efficiency of the matting process when interframe coding is used.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The scanline algorithms are popular in a computer graphics for complex geometric manipulations. The main characteristic of the scanline algorithms is that a geometric transformation is decomposed into multiples transforms with each pass operating only along row or column scanlines. This leads to conversion of 2D image manipulation problems to straightforward 1D problems resulting in simple and systematic methods. The goal of this work is to examine the scanline approach for manipulation of transform-compressed images without decompressing them. We show how the scanline algorithms for rotation and projective mapping can be developed for JPEG/DCT images. The performance of the proposed scanline algorithms is evaluated with respect to quality, speed, and control and memory overhead.

Multiple layer feedforward neural networks are often viewed as black boxes as the knowledge stored in the connection weights of these networks is generally considered incomprehensible. This paper suggests a solution to this deficiency of neural networks by proposing a backtracking tree search procedure for converting the weights of a neuron into a symbolic representation and demonstrating its use for understanding and symbolic approximation of feedforward neural networks. Several examples are presented to illustrate the proposed symbolic mapping of neurons.

Spatial image filtering is a method by which an image can be enhanced. The method involves computing correlation of a mask and input image to produce desired image. This requires a number of computations consisting of addition and division operations. Such operations are repeated multiple times for common areas of overlapping masks as we shift the mask over consecutive pixels. As a result, there is a significant number of redundant operations that slows down the spatial filtering process. In this paper, we propose a new method to eliminate redundant operations and we apply it to volumetric MRI images using mean filter. We analyse the complexity of the proposed method and compare its processing speed to that of the conventional implementation of the mean filter. Our method shows up to one order of magnitude improvement in processing speed over the conventional method especially on 3D images with large filter masks.

Many target tracking methods rely on the use of centroids as features points. In several applications such as the tracking of vehicles on roadways, the appropriate target paths are known a priori. Based on this observation, an efficient centroid computation method is presented and analyzed in this paper.

This paper presents a generalization of the correspondence approach of Sethi and Jain by extending the path coherence criterion to high dimensional vector space. This allows the same correspondence procedure to be used for a variety of tokens including points, lines, planes and regions. To demonstrate the generalized approach, we apply it to track lines and present experimental results.

The problem of determining the position and orientation of a mobile robot has been addressed by several researchers using sensors of different modalities, including video cameras. Invariably, all the vision-based approaches for robot localization consider that the camera is mounted on the robot and that the robot working environment is assumed to contain prominent landmarks at known locations. In this paper we propose a robot localization scheme where the robot itself serves as the landmark for cameras that are positioned in the environment to cover the entire work area of the robot. Although the proposed approach is applicable for the robots of any regular shape, we develop the solution to the localization problem by assuming a cylindrical shape for the robot. A compete mathematical analysis of the localization problem is given by extending the three-dimensional structure-from-rotational motion approach to the present task. We also examine the implementation issue of the proposed approach and present experimental results to show its effectiveness.

Decision tree classifiers represent a popular classification methodology that has been successfully used in many pattern recognition tasks. Noting many similarities between the tree classifiers and the multilayer perceptron classifiers, an artificial neural network based implementation of decision tree classifiers to enhance their performance is described here. Several issues related to decision tree classifiers are discussed and it is shown how these can be dealt with following the proposed implementation.

In addition to invariance with respect to certain geometric transformations, there are two other key requirements for any shape recognition system. It should be flexible enough to adopt to a variety of sets of shapes with minimal training; and it should be capable of performing even in presence of occlusion. This paper describes one such shape recognition system that is currently under development. The system is based on the redundant hashing scheme of Kohonen for recognizing and correcting misspelt words. The current version of the system is meant for 2-D shapes; however, the same approach is applicable to 3-D shape recognition. In the present implementation for 2-D shapes, a polygonal approximation of the given shape is encoded in the form of a string. The encoded string is then used to generate a very small set of shape hypotheses through the use of redundant hashing. The best hypothesis from the set of competing hypotheses is selected by a very simple matching scheme followed by a verification phase based on rotation transformation. The experiments thus far indicate that the system is capable of recognizing shapes in presence of occlusion with about 5% error. The system has significant ability to adopt to new sets of shapes; it does not require any training except the building of hash index table and the corresponding shape dictionary.

Echocardiographic images are generally of poor quality. From a strict gray scale perspective, there are few edges which correspond to the underlying anatomy. Trained ultrasonographers must use considerable a priori information regarding normal human anatomy and use this information extensively to 'fill in' the missing and scanty details of a typical echocardiogram. Due to these constraints, the conventional image processing approaches often fail miserably for echocardiographic images. Knowledge-based approaches for segmenting echo images have been proposed in the recent past; these appear to be much more successful. One of the limitations of knowledge-based approaches is that the acquisition of knowledge and the formation of rules is a relatively difficult task. Moreover, there are instances where experts may have different opinions; this is specially true in the context of echocardiographic images. Additionally, it is much more difficult to codify the expert's knowledge at the image level. Considering these factors plus the typical medical research scenario, where a series of images are acquired at one sitting for a patient, we present in this paper an approach for segmenting echocardiographic image sequences by directly acquiring the expert's knowledge at the image level. The present implementation yields a look up table capturing the expert's knowledge leading to near real time segmentation of echocardiographic images. The results obtained are quite in agreement with the segmentations obtained through trained ultrasonographers.

In this article, we report a study on the problem of person identification in TV programs, such as situation comedy shows. A person identification system is constructed based on the joint use of visual and audio information. The system consists of two modules, namely, the analysis and the fusion modules. The analysis module contains a visual analysis component responsible for detection, tracking, and recognition of faces in video, and the audio analysis component, which operates by speaker identification. Both components have their advantages under different circumstances and we studied how to exploit the interaction between them for improved performance. Two fusion strategies are compared in our research. In the first strategy, the audio-verify-visual fusion strategy, speaker identification is used to verify the face recognition result. The second strategy, the visual-aid-audio fusion strategy, consists of using face recognition and tracking to supplement speaker identification results. By comparing the output from our system with our ground truth database, we evaluate the performance of each individual analysis component and their fusion. The results show that while the audio-verify-visual fusion strategy has slightly lower recall than the original face recognition system, it achieves the best identification precision among different algorithms. This suggests that such a strategy is suitable for applications where precision is much more critical than recall (e.g., security systems). The visual-aid-audio fusion strategy, on the other hand, generates the best overall identification performance. It outperforms either of the individual analysis components greatly in both precision and recall. This strategy is suitable to more general applications, such as, in our case, person identification in TV programs.

This PDF file contains the editorial “Guest Editorial: Special Section on Image and Video Databases” for JEI Vol. 7 Issue 02

Scanline algorithms are popular in computer graphics for complex geometric manipulations. The main characteristic of scanline algorithms is that a geometric transformation is decomposed into multipass transforms with each pass operating only along row or column scanlines. This leads to conversion of 2-D image manipulation problems to straightforward 1-D problems resulting in simple and systematic methods. The goal of this work is to examine the scanline approach for manipulation of transform-compressed images without decompressing them. We show how the scanline algorithms for rotation and projective mapping can be developed for JPEG/DCT images. The performance of the proposed scanline algorithms is evaluated with respect to quality, speed, and control and memory overhead.

One of the challenging problems in video databases is the organisation of video information. Segmenting a video into a number of clips and characterising each clip has been suggested as one mechanism for organising video information. This approach requires a suitable method to automatically locate cut points (boundaries between consecutive camera shots in a video). Several existing techniques solve this problem using uncompressed video. Since video is increasingly being captured, moved, and stored in compressed form, there is a need for detecting shot boundaries directly in compressed video. The authors address this issue and show certain feature extraction steps in MPEG compressed video that allow the implementation of most of the existing cut detection methods developed for uncompressed video for MPEG video stream. They also examine the performance of three tests for cut detection by viewing the problem of cut detection as a statistical hypothesis testing problem. As the experimental results indicate, the statistical hypothesis testing approach permits fast and accurate detection of video cuts.

The authors propose two automatic threshold-selection schemes, based on functional approximation of the histogram. The first method is based on minimising the sum of square errors, and the second one is based on minimising the variance of the approximated histogram. Experimental results show that, on average, the latter scheme gives better results than the former one, at a small extra computational cost. A 'goodness' measure is proposed to measure the effectiveness of the two schemes, and to compare them against the entropy-based approach and the moment-based approach.

Shape discrimination using boundary information is well established and a number of methods are available to extract features from the boundary trace. In this paper a boundary scalar transform technique, wherein the pattern boundary is expressed in finite Walsh series expansion, is presented for the description and discrimination of polygonal curves. As the Walsh sequency spectra are not invariant to cyclic shift of the starting point on the boundary, two preprocessing methods have been suggested to normalize the starting point. A method is also presented for the reconstruction of the pattern boundary using Walsh descriptors. Finally, computer simulation results of experiments conducted to test the discriminatory power of Walsh descriptors and the effectiveness of the normalization of the starting point are presented and discussed.

One of the vital problems in motion analysis is to match a set of feature points over an image sequence. In this paper, we solve this problem by relying on continuity of motion which is known to play an important role in the perception of apparent motion in the human visual system. We propose a relaxation algorithm for feature point matching where the formation of smooth trajectories over space and time is favored. Experimental results on laboratory generated and real scene sequences are presented to demonstrate the merit of out approach.

In this paper a technique based on a charge analogy is presented for the design of edge detectors in various sizes. Using this analogy, an extended version of the Roberts cross operator is derived. The results of an edge-detection experiment on two digital pictures are also presented.

A nonparametric algorithm is presented for the hierarchical partitioning of the feature space. The algorithm is based on the concept of average mutual information, and is suitable for multifeature multicategory pattern recognition problems. The algorithm generates an efficient partitioning tree for specified probability of error by maximizing the amount of average mutual information gain at each partitioning step. A confidence bound expression is presented for the resulting classifier. Three examples, including one of handprinted numeral recognition, are presented to demonstrate the effectiveness of the algorithm.

A correspondence between the factored form representation of monotonic Boolean functions and the sequential evaluation procedures for them is shown to exist. Based on such a relationship, a criterion is developed for obtaining the cost of the sequential procedure directly from the factored form representation. Making use of this criterion, a simple algorithm is presented to determine the fast sequential evaluation procedure for monotonic Boolean functions.

A critical problem in automatic target recognition is the extraction and tracking of moving targets in high clutter, low resolution, low contrast thermal imagery. This problem is further exacerbated by the large distances that typically exist between the sensor and the targets. In this paper, a new paradigm called motion filtering is presented for exploiting motion to extract and track targets. The main feature of the present approach is the emphasis on the motion information. Experimental results from two sequences of tactical situations are presented to exemplify the capabilities of the proposed approach.

One of the vital problems in motion analysis is to match a set of feature points over an image sequence. In this paper, we solve this problem by relying on continuity of motion which is known to play an important role in motion perception in biological vision systems. We propose a relaxation algorithm for feature point matching where the formation of smooth trajectories over space and time is favored. Experimental results on a laboratory generated as well as a real scene sequence are presented to demonstrate the merit of our approach.

A method for extracting road boundaries using the monochrome image of a visual road scene is presented. The statistical information regarding the intensity levels present in the image along with some geometrical constraints concerning the road are the basics of this approach. Results and advantages of this technique compared to others are discussed. The major advantages of this technique, when compared to others, are its ability to process the image in only one pass, to limit the area searched in the image using only knowledge concerning the road geometry and previous boundary information, and dynamically adjust for inconsistencies in the located boundary information, all of which helps to increase the efficacy of this technique.

In this paper we present a method to segment a range image into regions which correspond to different object surfaces in the scene. We first obtain an equidistance contour map of the range image from slicing the range image at fixed increment distance values. Pixels along a contour are all at about the same distance from the sensor. We have observed that whenever a contour crosses an object surface edge, we would see direction discontinuity, curvature discontinuity, curvature zero-crossing, or termination of the contour. We call these places the critical points of the contour. We divide a contour into segments at its critical points. Next, we find the two corresponding contour segments on two consecutive slices. Every pair of corresponding contour segments defines a small region in the range image. Thus through registering contour segments in consecutive slices we have partitioned a range image into many small regions. Each region corresponds to a portion of an object surface. The last step is to merge these small regions into larger areas based on whether or not the corresponding scene surface segments of two adjacent regions have similar orientations in the 3-D space. The range image segmentation process is completed when the merging process is done. This approach is fast because it analyzes only the pixels along the equidistance contours and the entire process can be completed in just one pass.

extraction. This is in contrast to the early approaches of twoframe analysis. The use of long image sequences permitssome additional constraints like the continuity of motion tobe exploited provided that the frames are acquired at regularshort intervals of time. Motion analysis approaches usinglong image sequences are also more robust to noise due tothe redundancy of input data. This makes these approachesmore suitable for realistic applications.For motion analysis approaches using an extended imagesequence, it is very important that any motion discontinuityacross time be identified. This is independent of the goal ofmotion analysis, as such a detection of motion discontinuitywill permit the integration of desired information only fromthe appropriate part of the extended image sequence. As an

Recently we have proposed a range image segmentation method based on the equidistance contour map extracted from a range image. An equidistance contour of a range image is formed by pixels on the range image whose corresponding scene points are all at a same specified distance from the sensor. We have observed that in different ways the contours reflect the existence of object surface edges, the geometries of object surfaces, and the orientations of these surfaces in the 3-D space. In this paper we present a method which uses the equidistance contours to further characterize the regions obtained from segmentation. This is an integral part of an object recognition process. The meaning of characterization is to find the geometries of the corresponding 3-D object surfaces of the regions from the contours. If a surface has nice analytic form such as planar, spherical, cylindrical, or conical, we determine not only its type but also the values of the parameters which describe its geometry.

The goal of the paper is to present a novel Chi-square similarity measure and assess its performance through comparison with well-known similarity measures such as Cosine, Dice, and Jaccard. The Chi-square similarity measure has been designed to withstand the imperfections of transcribed spoken documents. The major difference of our similarity measure from others consists in the fact that in addition to searching for co-occurring words in documents, we also match informative closeness of common words. We assume that co-occurring words, which had been employed to convey the same information, should have the compatible significance in matching documents. To test it we apply the Chi-square method. Experimental results obtained via using an archive of transcribed news broadcasts demonstrate the high efficacy of the proposed methodology.