Search

In this work, an indoor sound field feature matching method is proposed and is applied to detect a mobile robot's location and orientation. The sound field feature, captured from a sound source to a pair of microphones, contains the dynamic of the propagation path. Because of the complexity of indoor environment, the features from different path can be distinguished using appropriate models. Gaussian mixture models are utilized in this paper to characterize the phase difference and magnitude ratio distributions between the microphone pair in consecutive data frames. The application provides an alternative thinking compared with traditional methods such as direction of arrival (DOA) using propagation delay. They usually suffer from reverberation, non-line-of-sight and microphone mismatch problems. The experimental results show the method not only has a high recognition rate for robot's location and orientation, but also is robust against environmental noise.

In this paper, the feasibility of utilizing spatial hearing for robot's pose search is investigated, and a novel and robust robot location and orientation detection method for highly complex and noisy environment is proposed. Unlike traditional methods, the proposed method imitates the behavior of human or animals that utilize the experience of sound field features at different location and orientation in a normal environment to locate themselves. Since this method can provide global location and orientation detection, it is suitable to navigate the robot from the present location to the destination by adjusting its orientation.

For intelligent robots to interact with people, an efficient human-robot communication interface is very important (e.g. voice command). However, recognizing voice command or speech represents only part of speech communication. The physics of speech signals includes other information, such as speaker direction. Secondly, a basic element of processing the speech signal is recognition at the acoustic level. However, the performance of recognition depends greatly on the reception. In a noisy environment, the success rate can be very poor. As a result, prior to speech recognition, it is important to process the speech signals to extract the needed content while rejecting others (such as background noise). This paper presents a speech purification system for robots to improve the signal-to-noise ratio of reception and an algorithm with a multidirection calibration beamformer.

This work proposes a novel and robust adaptive speech enhancement system, which contains both time-domain and frequency-domain beamformers using H/spl infin/ filtering approach in vehicle environments. A corresponding microphone array data acquisition hardware is also designed and implemented. Traditionally, mutually matched microphones are needed, but this requirement is not practical. To conquer this issue, the proposed system adapts the mismatch dynamics to allow unmatched microphones to be used in an array. Furthermore, to achieve a satisfactory speech recognition performance, the speech recognizer is usually required to be retrained for different vehicle environments due to different noise characteristics and channel effects. The channel effect usually causes the modeling error in a channel recovery process because of the long channel response. The proposed system using the H/spl infin/ filtering approach, which makes no assumptions about noise and disturbance, is robust to the modeling error. Consequently, the proposed frequency-domain beamformer provides a satisfactory performance without the need to retrain the speech recognizer.

This investigation proposes a robust robot localization system. The system contains a novel Gaussian mixture-sound field landmark model (GM-SFLM) and can localize the robot accurately in noisy environments. Moreover, the proposed method depends nothing on the geometry relation between source locations and two microphones; it is able to cover both near-field and far-field problems. With this proposed GM-SFLM, we can localize robot in 2-dimensional indoor environments. Furthermore, we realize the GM-SFLM into a quadruped robot system composed of an eRobot and a robot agent by using embedded Ethernet technology. The experiment demonstrates that when the robot is completely non-line-of-sight, this system still provides high detection accuracy. Additionally, the proposed method has advantages of high accuracy, low-cost, easy to implement and environmental adaptation.

This paper presents a real-time human-robot interface system (HRIS), which processes both speech and vision information to improve the quality of communication between human and an autonomous mobile robot. The HRIS contains a real-time speech attention system and a real-time face tracking system. In the speech attention system, a microphone-array voice acquisition system has been developed to estimate the direction of speaker and purify the speaker's speech signal in a noisy environment. The developed face tracking system aims to track the speaker's face under illumination variation and react to the face motion. The proposed HRIS can provide a robot with the abilities of finding a speaker's direction, tracking the speaker's face, moving its body to the speaker, focusing its attention to the speaker who is talking to it, and purifying the speaker's speech. The experimental results show that the HRIS not only purifies speech signal with a significant performance, but also tracks a face under illumination variation in real-time

Human-computer interaction (HCI) using speech communication is becoming increasingly important, especially in driving where safety is the primary concern. Knowing the speaker's location (i.e., speaker localization) not only improves the enhancement results of a corrupted signal, but also provides assistance to speaker identification. Since conventional speech localization algorithms suffer from the uncertainties of environmental complexity and noise, as well as from the microphone mismatch problem, they are frequently not robust in practice. Without a high reliability, the acceptance of speech-based HCI would never be realized. This work presents a novel speaker's location detection method and demonstrates high accuracy within a vehicle cabinet using a single linear microphone array. The proposed approach utilize Gaussian mixture models (GMM) to model the distributions of the phase differences among the microphones caused by the complex characteristic of room acoustic and microphone mismatch. The model can be applied both in near-field and far-field situations in a noisy environment. The individual Gaussian component of a GMM represents some general location-dependent but content and speaker-independent phase difference distributions. Moreover, the scheme performs well not only in nonline-of-sight cases, but also when the speakers are aligned toward the microphone array but at difference distances from it. This strong performance can be achieved by exploiting the fact that the phase difference distributions at different locations are distinguishable in the environment of a car. The experimental results also show that the proposed method outperforms the conventional multiple signal classification method (MUSIC) technique at various SNRs.

In this work, a robust speaker's location estimation method in a vehicle environment is presented. This method applies Gaussian mixture models (GMM) to the phase information obtained from a microphone array. The individual Gaussian component of a GMM represents some general location-dependent phase difference distribution between two microphones. These distributions are effective in modeling the speaker's location. The relation between geometry of microphone array and frequency band is taken into consideration to avoid aliasing problems. The proposed approach provides an accurate estimation even in near-field, noisy and complex vehicle environment. Moreover, it performs well not only in non-line-of-sight cases, but also in the conditions that the speakers are aligned in a direction to the microphone array with difference distances. Experiments are conducted in a mini-van vehicle and the results show that the proposed method outperform the popular technique multiple signal classification method (MUSIC) in different SNR cases.

In this paper, a self-calibrated speaker tracking system applied to both image tracking and sound source detection is proposed. The sound source estimated by microphone system is used to supply video tracking system. On the other hand, the direction of speaker detected by image tracking system is fed back to calibrate the phase distortion between microphones. Some experiments using actual devices are also done to test the performance of the proposed method. It is shown that significant improvement in source localization accuracy is possible when the camera and the microphone system are well calibrated.

This work presents a robust speaker's location detection algorithm using a single linear microphone array that is capable of detecting multiple speech sources under the assumption that there exist nonoverlapped speech segments among sources. Namely, the overlapped speech segments are treated as uncertainty and are not used for detection. The location detection algorithm is derived from a previous work (2006), where Gaussian mixture models (GMMs) are used to model location-dependent and content and speaker-independent phase difference distributions. The proposed algorithm is proven to be robust against the complex vehicular acoustics including noise, reverberation, near-filed, far-field, line-of-sight, and non-line-of-sight conditions, and microphones' mismatch. An adaptive system architecture is developed to adjust the Gaussian mixture (GM) location model to environmental noises. To deal with unmodeled speech sources as well as overlapped speech signals, a threshold adaptation scheme is proposed in this work. Experimental results demonstrate high detection accuracy in a noisy vehicular environment.