Search

Cybersickness is a known issue in virtual reality affecting a notable percentage of the populations. However, predicting the level and incidence of cybersickness in new systems is difficult. Past publications were analyzed for their factors and resulting cybersickness scores. These factors were then used to develop three predictive models using demographics, software, and hardware factors. Using demographic information alone explained 44.2% of the adjusted variance in a linear model. Using hardware and software factors alone explained 55.3% of the adjusted variance in a linear model. Using demographics, software, and hardware factors did not use a linear model, but rather had an average residual error of 1.03. This residual error is an estimate of how far the predicted cybersickness score is from the actual score.

The field of Sonification has led to a variety of projects aimed at conveying graphical information via audition. Most of these projects are aimed at specific domains or contexts, and few have studied general audio presentation techniques empirically. This paper details the results of a multipart study of user perception of selected audio stimuli as users are asked to interpret those stimuli as points in a generic Cartesian space. Results will be used in future work designing methods of portraying technical drawings such as flowcharts and UML via audio.

Purpose The purpose of this paper is to present the design and evolution of the Dancing Computer project. Dancing Computer is an ongoing research project at the Michigan State University, which is developing a system that aims to increase computer literacy in elementary-aged children by teaching them first to read code before they write it. The main objective is to educate children on basic concepts of computer science. Design/methodology/approach Children are given tablet computers that present a simple program line-by-line that they execute as they pretend to be a computer. The programs are acted out on a portable dance floor consisting of colored tiles, and the program statements instruct the child to move, turn and act out dance poses and terminology. Findings The Dancing Computer prototype was tested in six different locations in 2016, reaching approximately 250 students. Learning was demonstrated by significant improvements in both task duration and error performance as students performed the activities. The most common errors were movement errors, where participants failed to move the correct number of squares. Social implications This project has the potential to increase the level of computer literacy for thousands of children. This project’s goal is to increase understanding of what a computer does, what a program does and the step-by-step nature of computer programs. Originality/value This is a unique and a different approach – the norm being to start students off writing code in some language. In Dancing Computer stages children as readers of programs, allowing them to pretend to be a computer in a fun and engaging activity while also learning how computers execute real programs.

Cybersickness is an affliction common to users of virtual environments. Similar in symptoms to motion sickness, cybersickness can result in nausea, headaches, and dizziness. With these systems becoming readily available to the general public, reports of cybersickness have increased and there is a growing concern about the safety of these systems. This review presents the current state of research methods, theories, and known aspects associated with cybersickness. Current measurements of incidence of cybersickness are questionnaires, postural sway, and physiological state. Varying effects due to display and rendering modes, such as visual display type and stereoscopic or monoscopic rendering, are compared. The known and suspected application aspects that induce cybersickness are discussed. There are numerous potential contributing application design aspects, many of which have had limited study, but field of view and navigation are strongly correlated with cybersickness. The effect of visual displays is not well understood, and application design may be of greater importance.

Head Mounted Displays (HMDs) anecdotally exhibit a higher rate of cybersickness, motion-sickness like symptoms due to visual stimuli, than virtual reality systems that employ large screens. Yet, there have been relatively few multi-display studies, and frequently factors other than the display varied between conditions. We controlled for these additional factors in three experiments which considered HMDs weight, perceived screen size, and rendering mode. The HMD weight experiment had no effect on cybersickness, with p < 0.88. This is of benefit to HMD developers as it signifies that additional weight in the hardware is not cause for concern. While screen size and field of view (FOV) are often used interchangeably, the FOV has a strong effect independent of screen size. The perceived screen size experiment had no effect on cybersickness, with p < 0.66. This benefits cybersickness researchers, as it signifies that the results from monitor experiments can be compared directly with the results from large screen experiments, assuming the remaining factors are held constant. The rendering experiment, with stereo and mono rendering, had no effect on cybersickness, with p = 0.22. Since the HMD weight and screen size experiments used the same FOV, application, and navigation paradigm, they were compared for possible effects. Initially, we found a significant effect on cybersickness. After normalizing using the average change when a part of the real world is seen, such as in large screens, there is no longer a statistical difference (p < 0.77).

Geometry compression is the coding of 3D geometric data in a form that requires less space to store and less bandwidth to transmit. This 3D geometric data provides a computer graphics system with the scene description necessary to render images. Geometric data is quite large and, therefore, needs effective compression methods to decrease the transmission and storage requirements. A large amount of research has been focused on static geometry compression, but only limited research has addressed animated geometry compression, the compression of temporal sequences of geometric data. The goal of this paper is to represent 3D animation sequences with a reduced set of motion vectors that take advantage of the large data coherence in space and time. We proposed an octree-based motion representation method in which a small set of motion vectors are generated for each frame by analyzing the motion between consecutive frames. The vectors represent the differential motion from a previous frame to the current frame and are used to predict the vertex positions for the new frame. The octree motion representation process generates a hierarchical octree structure for each frame in the sequence. The octree approach can represent 3D animated sequences with high compression factors while maintaining reasonable quality. This approach is easy to implement and have a low cost encoding process and a fast decoding process, which make it very suitable for real time application.

Knowledge of objects, situations, or locations in the environment can be productive, useful, or even life-critical for mobile augmented reality (AR) users. Users may need assistance with (1) dangers, obstacles, or situations requiring attention; (2) visual search; (3) task sequencing; and (4) spatial navigation. The omnidirectional attention funnel is a general purpose AR interface technique that rapidly guides attention to any tracked object, person, or place in the space. The attention funnel dynamically directs user attention with strong bottom-up spatial attention cues. In a study comparing the attention funnel to other attentional techniques such as highlighting and audio cueing, search speed increased by over 50 percent, and perceived cognitive load decreased by 18 percent. The technique is a general three-dimensional cursor in a wide array of applications requiring visual search, emergency warning, and alerts to specific objects or obstacles, or for three-dimensional navigation to objects in space.

In 3 experiments, the authors investigated spatial updating in augmented reality environments. Participants learned locations of virtual objects on the physical floor. They were turned to appropriate facing directions while blindfolded before making pointing judgments (e.g., "Imagine you are facing X. Point to Y"). Experiments manipulated the angular difference between the learning heading and the imagined heading and between the actual heading and the imagined heading. The effect of actual-imagined on pointing latency was observed for naive users but not for users with brief training or instructions concerning the fact that objects can move with body movements. The results indicated that naive users used an environment-stabilized reference frame to access information arrays, but with experience and instruction the nature of the representation changed from an environment stabilized to a body stabilized reference frame.

We examined background characteristics of virtual reality participants in order to determine correlations to cybersickness. As 3D media and new VR display technologies from companies such as Occulus and Sony become more popular, the incidence of cybersickness is likely to increase. Understanding the impact of individual backgrounds on susceptibility can help shed light on which individuals are more likely to be impacted. Past history of motion sickness and video game play have the best predictive power of cybersickness of the factors studied. A model to estimate the likelihood of cybersickness using background characteristics is posed.

Parallel Text Alignment (PTA) is the automatic content alignment of multiple text documents originating or derived from the same source. Such an alignment allows for more powerful retrieval of document contents due to the addition of translation approaches to the retrieval problem. The implications of this result in improving multimedia data access in digital library applications, range from facilitating the analysis of multiple English language translations of classical texts to enabling on-demand and random comparison of multiple transcriptions derived from a given audio stream or associated with a given stream of video, audio, or images. In this paper we present an efficient algorithm for achieving such an alignment, and demonstrate its use with two applications. This result is an application of the new framework of Cross-Modal Information Retrieval recently developed at Dartmouth College and Michigan State University.

Creation of large and complex World Wide Web sites is hampered by the “page at a time” approach of many tools and the programming knowledge and custom software development required for automated solutions. This paper describes the development of Automatic Site Markup Language (ASML). ASML is a new markup language designed to automate and facilitate the production of large, complicated web sites which can include dynamic content or content that changes with time. ASML extends HTML with new, high-level features while still preserving complete compatibility with common browser and server technologies. It has powerful indexing and searching facilities, and enables the automatic translation of document formats. Most importantly, ASML provides HTML-like features at the site level rather than just the page level.

A wide variety of alternative speech-free input methods have been developed, including speech recognition, gestural commands, and eye typing. These methods are beneficial not only for the disabled, but for situations where the hands are preoccupied. However, many of these methods are sensitive to noise, tolerate little movement, and require it to be the primary focus of the environment. Morse code offers an alternative when background noise cannot be managed. A Morse code-inspired application was developed employing electromyograms. Several muscles were explored to determine potential electrode sites that possessed good sensitivity and were robust to normal movement. The masseter jaw muscle was selected for later testing. The prototype application demonstrated that the jaw muscle can be used as a Morse "key" while being robust to normal speech.

Theatre Engine is a three-phase project exploring the integration of mobile devices such as smart phones and tablet computers as user interaction mechanisms for live theatrical performances. This paper describes the technical details for the first performance in the series: Transitions, including system design, user interface details, and lessons learned. The user activities are described in the context of manipulations and the effects of those manipulations. The concept of blurring the distinction between real and virtual is explored. The prototype integrates Android-based mobile devices, servers, projection screens, and modern theatrical lighting with live musicians and dancers, allowing the audience to become a part of the presentation.

Widespread acceptance of virtual reality has been partially handicapped by the inability of current systems to accommodate multiple viewpoints, thereby limiting their appeal for collaborative applications. We are exploring the ability to utilize passive, untracked participants in a powerwall environment. These participants see the same image as the active, immersive participant. This does present the passive user with a varying viewpoint that does not correspond to their current position. We demonstrate the impact this will have on the perceived image and show that human psychology is actually well adapted to compensating for what, on the surface, would seem to be a very drastic distortion. We present some initial guidelines for system design that minimize the negative impact of passive participation, allowing two or more collaborative participants. We then outline future experimentation to measure user compensation for these distorted viewpoints.

In recent years there has been significant use of regular cross-polytopes (regular octahedrons or hyper-diamonds) as constructs to simplify problem solving in high-dimensional database queries, collision detection algorithms and graphic rendering techniques. Many of the algorithms for these applications use minimum volume bounding boxes as approximations of the polytopes to minimize computational complexity. The standard method [1] for finding these boxes in three dimensions uses the constraint of having two edges of a polyhedron coincident with two adjacent faces of the minimum bounding box. In this paper, we show that for a uniform cross-polytope in three dimensional space, a minimum volume bounding box would have a face flush with the convex hull of the polytope for all possible orientations of the polyhedron defined. We also show that the projections of the minimum bounding box of an n-dimensional regular cross-polytope are locally optimal with respect to the projections of the enclosed cross-polytope. We use this result to provide a necessary condition for the minimum bounding boxes of such polytopes. Finally, we show that if the two dimensional planar projections of a three dimensional uniform cross-polytope are simultaneously locally optimal then the polytope itself is optimally oriented.

Alzheimer's Disease (AD) is a neurological affliction that impacts primarily the aged due to brain tissue deterioration. It has been shown that this deterioration can be slowed down by engaging the person with daily interactive activities that include gaming, social interaction, memory exercises and physical activity. In this paper, we describe ZPLAY, a game-based user interface system which is designed to be web-based and to provide intervention therapy for AD. ZPLAY has two versions: the @lab version which is designed for diagnosis and used to measure different brain activation responses of AD and the @home version which is used to promote subject engagement and rehabilitation in a home environment in-between visits to the clinic.

Patients in the early stages of dementia are often subject to a variety of dangerous symptoms and behaviors including depression, wandering, and aggression. These conditions severely influence both quality of life and the degree of caretaker intervention that is necessary. Non-pharmacological interventions can alleviate these symptoms in some cases and are preferred as an alternative to medication when possible. This paper explores alternatives to the traditional definition of intervention, wherein the involvement of a caregiver is assumed, using a new modality wherein interventions are automatically instigated in response to conditions detected by physiological and behavioral measures. The environment is automatically manipulated in these methods in response to detected symptoms, intervening without the immediate assistance of a caregiver.

Human animation from motion capture data is typically limited to whatever movement was performed by the actor. A method to create a wider range of motion in the animation utilizes the motion capture database to synthesize new poses. This paper proposes a method to generate original natural poses based on the characteristics of natural poses based on motion capture data. Principal Component Analysis is used to transform the data into a reduced dimensional space. An unconstrained pose data set is created by calculating the position of the human skeleton based on the reduced dimensional space. Constrained pose data can be created using interpolation and iteration on the unconstrained pose data. We show some example results of the generated poses and compare these poses to poses created with iterative inverse kinematics methods. Results show that our method is more accurate and more natural than iterative inverse kinematics methods.

Two experiments were conducted to investigate whether locomotion to a novel test view would eliminate viewpoint costs in visual object processing. Participants performed a sequential matching task for object identity or object handedness, using novel 3-D objects displayed in a head-mounted display. To change the test view of the object, the orientation of the object in 3-D space and the test position of the observer were manipulated independently. Participants were more accurate when the test view was the same as the learned view than when the views were different no matter whether the view change of the object was 50° or 90°. With 50° rotations, participants were more accurate at novel test views caused by participants' locomotion (object stationary) than caused by object rotation (observer stationary) but this difference disappeared when the view change was 90°. These results indicate that facilitation of spatial updating during locomotion occurs within a limited range of viewpoints, but that such facilitation does not eliminate viewpoint costs in visual object processing.

The attention funnel is a general purpose AR interface technique that interactively guides the attention of a user to any object, person, or place in space. The technique utilizes dynamic perceptual affordances to draw user attention "down" the funnel to the target location. Attention funnel can be used to cue objects completely out of sight including objects behind the user, or occluded by other objects or walls.An experiment evaluating user performance with the attention funnel and other conventional AR attention directing techniques found that the attention funnel increased the consistency of the user's search by 65%, increased search speed by 22%, and decreased mental workload by 18%. The attention funnel has potential applicability as a general 3D cursor or cue in a wide array of spatially enabled mobile and AR systems, and for applications where systems can support users in visual search, object awareness, and emergency warning in indoor and outdoor spaces.

Knowledge of objects, situations, or locations in the environment may be productivity enhancing, generally useful, or even life-critical for mobile augmented reality (AR) users. Users may need assistance with: (1) dangers, obstacles, or situations requiring attention, (2) visual search, (3) task sequencing, and (4) spatial navigation. The omnidirectional attention funnel is a general purpose AR interface technique that rapidly guides attention to any object, person, or place in the space. The attention funnel dynamically draws user attention with a set of strong bottom-up spatial attention cues. In a study comparing the attention funnel to other attention directing techniques such as highlighting and audio cueing, we found that the attention funnel increased search speed by over 50%, and decreased perceived cognitive load by 18%. The technique has applicability as a general 3D cursor or cue in a wide array of AR applications that require visual search, emergency warning and alerts to specific objects or obstacles, or as a method for 3D navigation to objects in a room or guidance through space.

Optical see-through head-mounted displays (OSTHMDs) have many advantages in augmented reality application, but their utility in practical applications has been limited by the complexity of calibration. Because the human subject is an inseparable part of the eye-display system, previous methods for OSTHMD calibration have required extensive manual data collection using either instrumentation or manual point correspondences and are highly dependent on operator skill. This paper describes display-relative calibration (DRC) for OSTHMDs, a new two phase calibration method that minimizes the human element in the calibration process and ensures reliable calibration. Phase I of the calibration captures the parameters of the display system relative to a normalized reference frame and is performed in a jig with no human factors issues. The second phase optimizes the display for a specific user and the placement of the display on the head. Several phase II alternatives provide flexibility in a variety of applications including applications involving untrained users.

In this paper, we introduce QMR/PR, a flexible QoS-sensitive routing framework for mobile ad hoc networks, which addresses the problems caused by mobility-induced route failures. QMR/PR combines packet-interleaving using multipath routing and packet-level forward error correction. A particularly promising feature is its ability to significantly reduce both random and correlated packet losses, a critical issue for real-time applications. To provide accurate and reliable performance evaluation results, we have integrated QMR/PR into the QualNet simulation software, allowing us to conduct an extensive performance evaluation in a realistic networking architecture including all layers of the protocol stack. Simulation results demonstrate the effectiveness of the proposed framework and show that, compared to single-path routing, QMR/PR can reduce packet loss by 42-84 percent depending on the networking environment (e.g, node mobility, frequency of path failures), while simultaneously reducing routing overhead.

A new hybrid coding method for compressing 3D animated sequences of geometric data is presented. It is based on the most common representation of the geometric data: polygonal meshes for each discrete frame in the animation sequence. The method utilizes a delta coding and an octree-based method. It can achieve higher compression ratios than the octree-only method or the delta-only coding method. In this hybrid method, both the octree approach and the delta coding approach are applied to each single frame in the animation sequence in parallel. The approach that generates the smaller encoded file size is chosen to encode the current frame. Given the same quality requirement, the hybrid coding method is much better than either single approach in terms of compression ratio.

Although there has been great speculation about the potential of augmented reality (AR) in manufacturing applications, there have been very few empirical studies that assess the effectiveness of the technology. This chapter describes an experiment that tested the relative effectiveness of AR instructions in computer-assisted assembly. The AR solution displayed task information in the user’s field of view (FOV) as 3D objects registered with the workspace. The presentation demonstrates the exact execution of a procedural step. Three traditional instructional media approaches were compared with the AR system: a printed manual, computer-assisted instruction (CAI) using a monitor-based display, and CAI utilizing a head-mounted display (HMD). Results indicate that overlaying 3D instructions on the actual work pieces reduced the error rate for an assembly task by 82%, particularly diminishing cumulative errors, errors due to previous assembly mistakes. Measurement of the mental effort indicated decreased mental effort in the AR condition, suggesting some of the mental calculation of the assembly task is offloaded to the system. The results indicate that an AR system for computer-assisted assembly can improve worker performance.

Mobile ad hoc networks (MANETs) are an emerging class of network architectures that are characterized by their highly dynamic topology, limited resources (i.e., bandwidth and power), and lack of fixed infrastructure. The primary motivation for such networks is increased flexibility and mobility. Random node mobility along with various other factors such as network size and traffic intensity may be very dynamic, resulting in unpredictable variations in the overall network performance. This study centers on investigating and quantifying the effects of various factors and their two-way interactions on the overall performance of ad hoc networks. This study contributes to the modeling and development of adaptive ad hoc protocols (routing, medium access control, scheduling and buffer management). Using 2/sup k/r factorial experimental design, we isolate and quantify the effects of five factors: node speed, pause-time, network size, number of traffic sources, and type of routing (source versus distributed), that affect the performance of ad hoc networks. Specifically, this paper evaluates the impact of these factors on the following performance metrics: throughput, average routing overhead, and power consumption. Our study was conducted using a library-based simulator for sequential and parallel simulation of wireless networks.

Fiducial images are a common method for supporting vision-based tracking in augmented reality systems. This paper addresses the question: what is the best fiducial? A set of criteria that are desirable in an optically tracked fiducial are presented and a new fiducial image set is designed that meets these criteria. The images in this set utilize a square black-border pattern with a 15% border width and an interior image that supports orientation determination and unique identification. The interior image is constructed from orthogonal DCT basis images chosen to minimize the probability of misidentification and to be robust to noise and occlusion. We describe how this image can be integrated into an AR software system such as ARToolKit.

Although there has been much speculation about the potential of Augmented Reality (AR), there are very few empirical studies about its effectiveness. This paper describes an experiment that tested the relative effectiveness of AR instructions in an assembly task. Task information was displayed in user's field of view and registered with the workspace as 3D objects to explicitly demonstrate the exact execution of a procedure step. Three instructional media were compared with the AR system: a printed manual, computer assisted instruction (CAI) using a monitor-based display, and CAI utilizing a head-mounted display. Results indicate that overlaying 3D instructions on the actual work pieces reduced the error rate for an assembly task by 82%, particularly diminishing cumulative errors - errors due to previous assembly mistakes. Measurement of mental effort indicated decreased mental effort in the AR condition, suggesting some of the mental calculation of the assembly task is offloaded to the system.

A crucial aspect in the implementation of an augmented reality (AR) system is determining its accuracy. The accuracy of a system determines the applications it can be used for. The aim of our research is measuring the overall accuracy of an arbitrary AR system. Once measurements of a system are made, they can be analyzed for determining the structure and sources of errors. From the analysis it may also be possible to improve the methods used to calibrate and register the virtual to the real. This paper describes an online system for measuring the registration accuracy of optical see-through augmentation. By online, we mean that the user can measure the registration error they are experiencing while they are using the system. We overcome the difficulty of not having retinal access by having the user indicate the projection of a perceived object on a planar measurement device. Our method provides information which can be used to analyze the structure of the system error in two or three dimensions. The results of the application of our method to two monocular optical see-through AR systems are shown.

This paper describes a keyframe summarization method for client-server applications. This technique is designed for applications where a camera is collecting content on a continuous basis that must be transmitted in a summarized form to a remote database server over wireless network. The system combines three keyframe selection methods including a novel fast motion-based selection method, keyframe pooling and clustering for bandwidth control, and network bandwidth estimation.

Current Video on Demand (VoD) systems do not provide for interactive VCR-like scan operations such as rewind or fast- forward. Pre-storing the entire movie on a set-top box for implementation of such operations is currently neither practical nor feasible. Other approaches to VCR-like controls are either computationally complex or the server or consume large amounts of bandwidth during the operation. This paper proposes a novel technique, VVD, for real-time implementation of VCR-like operations, supporting multiple scan speeds, for Video on Demand with low latency and overhead. VVD uses multi-level compression of movie files to encode the same movie at a hierarchy of playout rates. Depending on the rate at which a scan operation is requested, the corresponding compressed movie file is transmitted. Frame dropping and repetition are used to provide smooth speed transitions among the multiple levels. The overhead of this scheme can be controlled to be less than 100% of the movie storage size.

Lectures and similar presentations are increasingly available in digital format as tools for presenting and recording become rich in features and readily available. We discuss extensions to a representative of such tools that facilitate better retrieval functionality while keeping the system simple and the instructor focused on the presented material, rather than on adding the necessary meta- information by hand.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

List of Figures. List of Tables. Preface. Acknowledgments. 1. Introduction. 2. Media, Multimedia, and Synchronization Models. 3. Multiple Media Correlation. 4. Algorithms for Multiple Media Correlation. 5. Parallel Text Alignment. 6. Text-To-Speech Alignment. 7. The Xtrieve System. 8. Applications and Future Work. Appendices. Index.

Computed multimedia synchronization and multiple media correlation are new areas of research in multimedia data analysis. As such, the applications and solutions presented in this book are the Genesis of this new research area. This chapter describes many potential applications for multiple media technologies, as well as possibilities for core research in multiple media analysis and computed synchronization. This chapter is representative of planned future work in this field. Some of these applications are already in preliminary development at the DEVLAB and the Michigan State University Media and Entertainment Technologies Laboratory (METLAB), while others represent future research areas proposed or planned.

Chapter 3 presented a general model for multiple media correlation. This chapter presents a specific algorithmic solution strategy for that model. As presented, multiple media correlation is a very general model and a single solution approach encompassing all possible applications is not practical. However, the solution approach presented in this chapter is appropriate for large classes of problems, namely those which can be expressed using media representation graphs (MRGs). Media representation graphs are discussed in Section 5..

This book develops a new framework for computed synchronization and demonstrates it within the context of several applications. A major contribu­tion of this research is the development of a cross-modal information retrieval (CMIR) system that can serve as a test-bed for application of computed synchro­nization technologies: the Xtrieve cross-modal information retrieval system. Xtrieve provides infrastructure in support of media alignment technologies and is the implementation environment for the solutions presented in Chap­ters 5 and 6. Xtrieve also provides a tool for exploring the use of computed synchronization technologies in information retrieval applications.

This chapter develops general models that form the framework for computed synchronization solutions. The goal of computed synchronization is the automatic determination of causal and spatial synchronization among media objects. The models that are developed provide a basis for the construction of solutions to real-world problems in media synchronization, as well as a basis for proving solution validity. Techniques based on these models are collectively referred to as multiple media correlation (MMC). Chapter 4 presents algorithmic approaches to solving these models.

Text-to-speech alignment is the alignment of a textual transcript to an audio stream. The computed synchronization data is a mapping from words in the text transcript to temporal intervals in the audio. This alignment provides a basic tool that facilitates many applications and has wide-spread general applicability.

Parallel Text Alignment (PTA) is the problem of automatically aligning content in multiple text documents originating or derived from the same source. The implications of this result in improving multimedia data access in digital library applications range from facilitating the analysis of multiple English language translations of classical texts to enabling the on-demand and random comparison of multiple transcriptions derived from a given audio stream, or associated with a given stream of video, audio, or images. In this paper we give an efficient algorithm for achieving such an alignment, and demonstrate its use with two applications. This result is an application of the new framework of Cross-Modal Information Retrieval recently developed at Dartmouth.

ImageTcl is an new system which provides powerful Tcl/Tk based media scripting capabilities similar to those of the ViewSystem and Rivl in a unique environment that allows rapid prototyping and development of new components in the C++ language. Powerful user tools automate the creation of new components as well as the addition of new data types and file formats. Applications using ImageTcl at the Dartmouth Experimental Visualization Laboratory (DEVLAB) include multiple stream media data analysis, automatic image annotation, and image sequence motion analysis. ImageTcl combines the high speed of compiled languages with the testing and parameterization advantages of scripting languages.

This paper describes an approach to the problem of searching speech-based digital audio using cross-modal information retrieval. Audio containing speech (speech-based audio) is difficult to search. Open vocabulary speech recognition is advancing rapidly, but cannot yield high accuracy in either search or transcription modalities. However, text can be searched quickly and efficiently with high accuracy. Script- light digital audio is audio that has an available transcription. This is a surprisingly large class of content including legal testimony, broadcasting, dramatic productions and political meetings and speeches. An automatic mechanism for deriving the synchronization between the transcription and the audio allows for very accurate retrieval of segments of that audio. The mechanism described in this paper is based on building a transcription graph from the text and computing biphone probabilities for the audio. A modified beam search algorithm is presented to compute the alignment.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The advent of modern electronic multimedia publishing and delivery systems extends previous modes of publishing to incorporate new media and new features. For example, a growing number of conferences, particularly in the field of Computer Science, are creating electronic proceedings that are made available on CD-ROM or via networks and are extending the proceedings to incorporate non-textual materials, such as video or audio segments, and new features, such as retrieval mechanisms and hypertext cross-linking. In this paper we discuss interactive multimedia publications (IMMpubs), drawing on our experience creating such electronic conference proceedings and other multimedia publications. Our discussions emphasize the continuing role of text, highlight the pitfalls of some publication systems, and illustrate the potentials of interactive multimedia publications.

Multimedia systems integrate text, audio, video, graphics, and other media and allow them to be utilized in a combined and interactive manner. Using this exciting and rapidly developing technology, multimedia applications can provide extensive benefits in a variety of arenas, including research, education, medicine, and commerce. While there are many commercial multimedia development packages, the easy and fast creation of a useful, full-featured multimedia document is not yet a straightforward task. This paper addresses issues in the development of multimedia documents, ranging from user-interface tools that manipulate multimedia documents to multimedia communication technologies such as compression, digital video editing and information retrieval. It outlines the basic steps in the multimedia authoring process and some of the requirements that need to be met by multimedia development environments. It also presents the role of video, an essential component of multimedia systems and the role of programming in digital video editing. A model is described for remote access of distributed video. The paper concludes with a discussion of future research directions and new uses of multimedia documents.

It is commonly accepted that the modification of the weights during training of an Artificial Neural Network can be augmented by addition of a random element chosen from various distributions. This technique, referred to as Noise Injection, allows the training process to stochastically traverse a larger subset of the sample space, as well as escape from local minima. This paper examines the effect of noise injection on the training cycle of feedforward neural networks. Emphasis is placed on the gradient descent weight modification technique of the backpropagation model. Statistical examination is made of the distribution of the effect within the topology of the weight space, upon the inputs to individual units, upon training time, and on the total error of the network. Since the weights of the network can be considered together as an n-tuple, injection of noise can be statistically examined within that n- dimensional space. It is shown that, for stochastically independent random distributions, the effect on this weight space and on the inputs to individual units is dependent upon the number of weights in the network. The multivariate distribution of the vector modification during training becomes increasingly distorted as the network size increases, such that noise injection has a more significant, and less stable, effect. Problems with traditional approaches are examined and an alternative noise injection method based on network size is presented.© (1993) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The Backpropagation technique for supervised learning of internal representations in multi4ayer artificialneural networks is an effective approach for solution of the gradient descent problem. However, as a primarilydeterministic solution, it will attempt to take the best path to the nearest minimum, whether global or local. If alocal minimum is reached, the network will fail to learn or will learn a poor approximation of the solution. Thispaper describes a novel approach to the Backpropagation model based on Simulated Annealing. This modifiedlearning model is designed to provide an effective means of escape from local minima. The system is shown toconverge more reliably and much faster than traditional noise insertion techniques. Due to the characteristics of the cooling schedule, the system also demonstrates a more consistent training profile. 1. BACKGROUNDOne of the most popular learning models for artificial neural networks today is the backpropagation model (BP)14. The BP model consists of nodes (referred to as neurons) and interconnections (weights). Learning consists