Search

A study was conducted to determine the benefits of adding voice input to the graphical user interface (GUI) of a geometry learning software (Cabri-geometry II). In particular, we aimed to test whether or not voice input has an advantage over mouse input, regarding speed and user's preference. An experiment was conducted on 8 students aged between 15 and 16 who interacted with both the standard interface (mouse/keyboard) of the software and a multimodal interface in which speech recognition was simulated by the "Wizard of Oz" technique. The results show that voice input is significantly faster than mouse/keyboard input for command completion and that students prefer using voice commands rather than mouse/keyboard or multimodal commands.

We propose a new human interface (HI) system named “GazeToTalk” that is implemented by vision based gaze detection, acoustic speech recognition (ASR), and animated human-like agent CG with facial expressions and gestures. The “GazeToTalk” system demonstrates that eye-tracking technologies can be utilized to improve HI effectively by working with other non-verbal messages such as facial expressions and gestures.Conventional voice interface system have the following serious drawbacks. (1) They cannot distinct between input voice and other noise, and (2) cannot understand who is the intended hearer of each utterance. A “push-to-Wk” mechanism can be used to ease these problems, but it spoils the advantages of voice interfaces (e.g. contact-less, suitability in hand-busy situation).In real human dialogues, besides exchanging content messages, people use non-verbal messages such as gaze, facial expressions and gestures to establish or maintain conversations, or recover from problems that arise in the conversation.The “GazeToTalk” system simulates this kind of “meta-communication” facility by utilizing vision based gaze detection, ASR, and human-like agent CG. When the user intends to input voice commands, he gazes on the agent on the display in order to request to talk, just as in daily human-human dialogues. This gaze is recognized by the gaze detection module and the agent shows a particular facial expression and gestures as a feedback to establish an “eye-contact.” Then the system accepts or rejects speech input from the user depending on the state of the “eye-contact.”This mechanism allows the “GazeToTalk” system to accept only intended voice input and ignore another voices and environmental noises successfully, without forcing any arbitrary operation to the user. We also demonstrate an extended mechanism to treat more flexible “eye contact” variations.The preliminary experiments suggest that in the context of meta-communication, nonverbal messages can be utilized to improve HI in terms of naturalness, friendliness and tactfulness.

Auditory presentation methods may significantly enhance the interaction quality during user-computer dialogue. The impact of auditory interaction methods is important in the context of non-visual interaction, where audio is the primary direct perception output modality. In a few cases, 3D-audio output techniques have been employed for providing interaction for blind users. Unfortunately, such developments have been too specialized and do not support re-usability of the implemented approaches and techniques in different contexts, where non-visual interaction needs to be realized. A generic re-usable environment has been implemented, based on 3D audio, 3D pointing, hand gestures and voice input, which is applicable in all cases that interactive hierarchically structured selections from sets of alternatives must be handled. This environment has been used to implement the hierarchical navigation dialogue in a multi- media non-visual toolkit currently under development. It is composed of a set of modules implementing re-usable functionality with which interaction for non-visual hierarchical navigation can be realized within any non-visual interaction toolkit.

A technique that measures the impact of command selection on task time and errors is described. Users were timed while performing a drawing task, then while performing the same task with interpolated command selections. The difference between these times, consisting of both the time to select the command and to resume drawing, is the time cost of command selection. Several interface configurations were evaluated with this method including selected combinations of single mouse, two mice, voice and touch. Touch and voice input resulted in faster command selection times (approximately 1 sec) than any of the mouse conditions (approximately 3 sec).

The performance of two speech recognition systems installed at two field sites was analyzed. The speech systems were part of larger computer systems that were performing real functions in industrial environments. The two sites appeared to be polarized in terms of expected suitability for speech recognition. The variables looked at included task complexity, memory load, requirements for verification and error correction, vocabulary and syntax, microphone, operator experience and complexity of host computer software. Accuracy and throughput were measured for the speech recognition system at each site. The same measurements were made for keyboard entry. Operator differences account for most of the variance in results. Accuracy with voice input was higher than with keyboard for most operators. The most accurate operators with keyboard also tended to be the most accurate with voice. Throughput data appears more sensitive to individual differences in dealing with voice input, although the throughput data was clouded by slow host system response times overall. The discussion suggests that one to one replacement of keyboard with voice overlooks some possible advantages of voice. It is also possible to find operators who work well with voice. For those who do not work well with voice, the problems appear to be related to general work habits and attitude, rather than to specific difficulties with speech.

Recent technological advances in connected-speech recognition and position sensing in space have encouraged the notion that voice and gesture inputs at the graphics interface can converge to provide a concerted, natural user modality.The work described herein involves the user commanding simple shapes about a large-screen graphics display surface. Because voice can be augmented with simultaneous pointing, the free usage of pronouns becomes possible, with a corresponding gain in naturalness and economy of expression. Conversely, gesture aided by voice gains precision in its power to reference.

An experiment was run in which elderly and younger people used a keyboard editor and a simulated listening typewriter to compose letters. Performance was measured and participants rated the systems used. Our general conclusions were as follows: There are no major differences in performance between elderly computer users and their younger counterparts in carrying out a computer-based composition task. Elders appear to be more enthusiastic users of computer systems than are younger people. This is shown by preference ratings, behavioral observations, and post-experimental debriefings. Voice input does not improve performance on composition tasks, but it is greatly preferred over the traditional keyboard input method.

Richard Rabin Interactive graphics systems typically require intense “hands busy/eyes busy and brains busy” activity on the part of the system user/operator. Voice input by means of automatic speech recognition equipment, offers major potential for improving user/operator productivity. It is the only input technique which does not require the direct use of hands and eyes. Voice input can replace or complement keyboards, function keys, tablets and other types of input devices typically employed for entering commands, alpha and numeric data. Alan R. Strass Effective human interfaces are an essential element of plant information systems and computer integrated processes, such as graphics. In the past, data gathering choices in the factory have been generally dominated by clipboards and travelling punched card decks. Similarly, complicated keystroke sequences have often been required to evoke appropriate computer controlled functionality. Today, a number of advanced human interface techniques can be used to improve both source data capture and the selection of appropriate computer controlled operations. These techniques are becoming an integral part of many emerging on-line/real-time engineering and manufacturing applications. Speech recognition, in particular, is emerging as an important interface technology. Speech input can reduce the amount of attention the user has to spend on the mechanics of recording information of selecting functions and allows users to concentrate on their primary task. Some of the benefits include: (1) reduced user training time, (2) increased worker productivity, (3) reduced secondary key input, and (4) improved timeliness and accuracy of information made available via voice. Currently available speech recognition products have already been used to demonstrate these benefits. These advantages will only increase as the next generation of speech products deliver improved recognition performance. Mark Robillard There is an increasing awareness of the potential for improving operator productivity for graphic systems by the use of voice input. We have analyzed a number of applications in conjunction with potential users to determine how best to use this capability. Initial conclusions are that voice input can be effective, providing that the capabilities of the speech recognition equipment utilized to input voices are matched to the requirements of the applications. Key factors to be considered are vocabulary size and types, and the use of isolated words versus continuous speech utterances. Sue Schedler The addition of voice input to Calma's GBSll System provides users with a fast, accurate means by which they can execute commands. A customized Interstate Electronic VRM System has been integrated into the hardware configuration of all microelectronics products, Chips, Sticks and CARDSII. System software recognizes the input from the Harmony VRM in the same manner as a keyboard or menu button input. All three input methods may be used separately or in conjunction with each other. All command inputs are buffered, so the user need never wait for a command to complete before entering the next. A voice file contains up to 50 words, and any number of files may be uploaded or downloaded by one user. Both standard system commands or GPLII Programs can be executed by voice. On-line design productivity by menu input can be increased by 50% when the operator uses voice. Operator training is unaffected by the addition of voice to the system. Voice is trained as a separate module in the class. Its greatest impact appears once the users know operation syntax and can communicate commands quickly. Calma provides standard voice menu to be used by customers or the customer may modify the file as needed. Most commonly used commands are put into the voice file leaving less frequently used commands for the onscreen menu. By implementing the combination of voice and menu, we can completely eliminate the need for keyboard input. Operators initially are skeptical of the “bell and whistle” feature, but find after using voice input that they do not feel uncomfortable “talking” to the computer, and find it an enjoyable and productive tool. Matthew Peterson The use of voice recognition in a “mature” graphics application raises a number of issues that must be faced. “Mature” refers specifically to the maturity of the user interface in the graphics application. Commercial CAD/CAM systems provide such an application: they have been evolving for over ten years, they possess robust user interface features, and their features are subject to the test of the marketplace. Use of voice recognition in such applications raises a number of challenges to the technology: (1) How does it compare to existing features (e.g., tablet menuing) for servicing typical user interface needs? (2) Are there atypical or emerging needs for which it is particularly suitable? (3) How attractive is its price/productivity offering compared to alternative user investment strategies for increasing productivity? (4) In what directions should it be driven to better service the needs of commercial graphics applications? Material on present and future industry needs, other user interface features, and industry price trends will be presented to help assess these issues.

This paper describes a practical synthetic visual environment for use in CAD and teleoperation. Instead of using expensive head mounted display systems, we use a standard display and compute smooth shaded images using an AT&T Pixel Machine. The interface uses a VPL DataGlove [9] to track the hand, bringing the synthetic world into the same space as the hand. Hand gesturing is used to implement a virtual control panel, and some 3D modeling tasks. When simple speech recognition was added it markedly improved the interface. We also outline what extensions might be needed for using this kind of interface for teleoperation.