Search

The role of robot technologies is growing particularly in the areas of disaster response, infrastructure inspection, and so on. For remote operation of the robots, wireless communication is importa...

This paper addresses the problem of 2D sound source localization using multiple microphone arrays in an outdoor environment. Two main issues exist in such localization. Since the localization perfo...

Disaster robotics, in particular, unmanned aerial vehicle (hereinafter, drone) is expected to improve the promptness and effectiveness of search and rescue mission. Although the most common sensor for drones is vision, its drawbacks such as ill-illumination condition and occlusion should be compensated by other modalities. This paper focuses on sound source localization and position estimation by drone audition. The most critical issue of robot audition with a microphone array is the suppression of ego-noise caused by rotors and airflow around the drone. Another issue is handling of multiple sound sources in addition to ego-noise. If multiple sound sources are actually or virtually crossing due to movement of sound sources and/or the drone, tracking may suffer from uncertainty in data association. In addition, user interface is also critical for the mission, which has not been discussed to date. This paper focuses on the design and implementation of real-time visualization of sound source positions for search and rescue mission. The design follows the guideline proposed by Murphy and Tadokoro. As a proof of concept, public demonstrations at ImPACT Tough Robotics Challenge (TRC) are reported.

[abstFig src='/00290001/15.jpg' width='300' text='Visualization of localization result' ] We have studied on robot-audition-based sound source localization using a microphone array embedded on a UAV (unmanned aerial vehicle) to locate people who need assistance in a disaster-stricken area. A localization method with high robustness against noise and a small calculation cost have been proposed to solve a problem specific to the outdoor sound environment. In this paper, the proposed method is extended for practical use, a system based on the method is designed and implemented, and results of sound source localization conducted in the actual outdoor environment are shown. First, a 2.5-dimensional sound source localization method, which is a two-dimensional sound source localization plus distance estimation, is proposed. Then, the offline sound source localization system is structured using the proposed method, and the accuracy of the localization results is evaluated and discussed. As a result, the usability of the proposed extended method and newly developed three-dimensional visualization tool is confirmed, and a change in the detection accuracy for different types or distances of the sound source is found. Next, the sound source localization is conducted in real-time by extending the offline system to online to ensure that the detection performance of the offline system is kept in the online system. Moreover, the relationship between the parameters and detection accuracy is evaluated to localize only a target sound source. As a result, indices to determine an appropriate threshold are obtained and localization of a target sound source is realized at a designated accuracy.

[abstFig src='/00290001/22.jpg' width='300' text='Spatial-cue-based probabilistic model' ] This paper addresses bird song scene analysis based on semi-automatic annotation. Research in animal behavior, especially in birds, would be aided by automated or semi-automated systems that can localize sounds, measure their timing, and identify their sources. This is difficult to achieve in real environments, in which several birds at different locations may be singing at the same time. Analysis of recordings from the wild has usually required manual annotation. These annotations may be inaccurate or inconsistent, as they may vary within and between observers. Here we suggest a system that uses automated methods from robot audition, including sound source detection, localization, separation and identification. In robot audition, these technologies are assessed separately, but combining them has often led to poor performance in natural setting. We propose a new Spatial-Cue-Based Probabilistic Model (SCBPM) for their integration focusing on spatial information. A second problem has been that supervised machine learning methods usually require a pre-trained model, which may need a large training set of annotated labels. We have employed a semi-automatic annotation approach, in which a semi-supervised training method is deduced for a new model. This method requires much less pre-annotation. Preliminary experiments with recordings of bird songs from the wild revealed that our system outperformed the identification accuracy of a method based on conventional robot audition.**This paper is an extension of a proceeding of IROS2015.

This paper addresses online outdoor sound source localization using a microphone array embedded in an unmanned aerial vehicle (UAV). In addition to sound source localization, sound source enhancement and robust communication method are also described. This system is one instance of deployment of our continuously developing open source software for robot audition called HARK (Honda Research Institute Japan Audition for Robots with Kyoto University). To improve the robustness against outdoor acoustic noise, we propose to combine two sound source localization methods based on MUSIC (multiple signal classification) to cope with trade-off between latency and noise robustness. The standard Eigenvalue decomposition based MUSIC (SEVD-MUSIC) has smaller latency but less noise robustness, whereas the incremental generalized singular value decomposition based MUSIC (iGSVD-MUSIC) has higher noise robustness but larger latency. A UAV operator can use an appropriate method according to the situation. A sound enhancement method called online robust principal component analysis (ORPCA) enables the operator to detect a target sound source more easily. To improve the stability of wireless communication, and robustness of the UAV system against weather changes, we developed data compression based on free lossless audio codec (FLAC) extended to support a 16 ch audio data stream via UDP, and developed a water-resistant microphone array. The resulting system successfully worked in an outdoor search and rescue task in ImPACT Tough Robotics Challenge in November 2016.

This chapter contains from Sects. 3.1 to 3.5. Section 3.1 describes firstly the definition of drones and recent trends. The important functions of the search and rescue flying robot are also generally described. And, Sect. 3.1 consists of an overview of R&D technologies of flying robot in Tough Robotics Challenge and a technical and general discussion about a future prospective of flying robot including the real disaster survey and technical issues. Namely, drones or unmanned aerial vehicles (UAVs) should be going to real and bio-inspired flying robot.

This paper addresses 2D sound source localization using multiple microphone arrays with the future aim to be applied to outdoor environments. To achieve such sound source localization, there are two main issues to overcome. One is the lack of knowledge about how to design the system, since the localization performance depends on resolution of the array’s Transfer Function, the number of used microphone arrays, signal-to-noise ratio and the target sound’s frequency characteristics. We conduct a thorough analysis of the localization results with different simulation conditions. The other problem is the distinction between multiple sound sources, which is directly related to the appearance of outliers in the localization process of these sounds. To solve this issue, we propose an outlier removal method, taking sound property of the observed sounds into consideration. As the application scenario of the simulations, we selected bird song analysis, which provides a challenging environment in terms of constantly changing signal-to-noise ratio and relative sensor-to-target position. When the task is automated, robust robot audition technology can be established, possibly advancing one step closer to creating a fully automated bird song analysis system. We established a prototype system using the proposed method. We present several simulation results followed by a discussion on the issues, leading to establishing system design guidelines that ensure a predictable performance.

This paper addresses real-time bird song scene analysis. Observation of animal behavior such as communication of wild birds would be aided by a portable device implementing a real-time system that can localize sound sources, measure their timing, classify their sources, and visualize these factors of sources. The difficulty of such a system is an integration of these functions considering the real-time requirement. To realize such a system, we propose a cascaded approach, cascading sound source detection, localization, separation, feature extraction, classification, and visualization for bird song analysis. Our system is constructed by combining an open source software for robot audition called HARK and a deep learning library to implement a bird song classifier based on a convolutional neural network (CNN). Considering portability, we implemented this system on a single-board computer, Jetson TX2, with a microphone array and developed a prototype device for bird song scene analysis. A preliminary experiment confirms a computational time for the whole system to realize a real-time system. Also, an additional experiment with a bird song dataset revealed a trade-off relationship between classification accuracy and time consuming and the effectiveness of our classifier.

We have been studied about non-contact measurement of respiration and heart rates. In previous papers, the measurement system of small velocity using the M-sequence-modulated signal and phase difference of reflected signals from the target has been proposed. In this paper, we describe measurement of breathing and heartbeat in a standing position using the proposed method. Body-surface velocities by breathing and heartbeat could be observed respectively when the volunteer was breathing and holding the breath. In addition, measured velocity of breathing volunteer includes the component by heartbeat. It is considered that it has possibility to measure breathing and heartbeat concurrently.

This paper addresses bird song scene analysis focusing on location of birds and acoustic features of bird songs. Such a research area usually requires manual annotation related to positions and/or vocalization types of the target animals for a large amount of observed data. However, this manual annotation has two problems. One is that it is tough to annotate data observed in real environments because environmental noise exist and sound is reflected by trees and the ground, and also several birds at different locations may sing at the same time. The other is that it is inevitable that manual annotation produces inaccurate and inconsistent labels due to human errors and annotators' individual differences. For the first problem, we propose a Spatial-Cue-Based Probabilistic Model (SCBPM), which is a probabilistic model to estimate the maximum likelihood result for a bird song scene analysis by integrating sound source detection, localization, separation and identification based on spatial information of sound sources. For the second problem, we employ a semiautomatic annotation approach, in which a semi-supervised training method is deduced for SCBPM. This method decreases the amount of manual annotation. Preliminary experiments using recorded bird song data from the wild revealed that our system outperformed a conventional bird song scene analysis system by simply connecting sound source detection, localization, separation and identification in a cascade way in terms of identification accuracy.

Because of a growing need for unconstrained medical monitoring for unobtrusively observing individuals in a living space, we have been studied about non-contact measurement of vital information such as respiration and heartbeat using airborne ultrasound. In previous study, the measurement system of small displacement using the M-sequence-modulated signal and tracking phase difference of reflected signals from the target has been proposed. The measurement of respiration and heartbeat of the target in a standing position using a pair of the loudspeaker and the microphone has also been performed. However, body position cannot be measured using a pair of the loudspeaker and the microphone because the system can measure the distance to the body only. In this paper, we describe a basic study of the measurement of body position, respiration, and heartbeat. In the system, using microphone array, body position was estimated by synthetic aperture processing. In addition, small body-surface velocity by respiration a...

The non-contact measurement of the propagation characteristics in the human body using pass-through airborne ultrasound has been proposed. Detection and evaluation of airborne ultrasound passed through the heel have been reported. However, estimation of the speed of sound (SOS) in the heel is difficult due to large attenuation of the pass-through ultrasound. In this paper, airborne ultrasound passed through the wrist is studied. The pass-through ultrasound can be detected by pulse compression using more than 16th-order M-sequence. Attenuation of the pass-through ultrasound or the estimated SOSs in the wrist are different by the transducer position. When the transducers are set at the center of the wrist, attenuation and the SOS are 86.2 dB and 1780 m/s, respectively.

The elastic properties of human tissues can be quantitatively evaluated from the ultrasonic propagation speed in tissues. To effectively propagate ultrasound in human tissues, ultrasonic transducers are typically brought into contact with tissue surfaces. In this study, the non-contact evaluation of human tissues using pass-through airborne ultrasound has been proposed. When airborne ultrasound propagates and passes through tissues, the pass-through wave is extremely attenuated. To detect the attenuated pass-through wave in the received signal, the signal-to-noise ratio (SNR) of the received signal is improved by pulse compression using a higher-order M-sequence in the proposed method. In this paper, the estimation of ultrasonic propagation speeds in tissue-mimicking phantoms is described. The urethane-rubber phantom and solutions of ethanol in water are used as the phantoms. The time of flight (TOF) of the pass-through wave in the phantom is determined from the wave front. The propagation speed in the phantom is estimated using the determined TOF. Propagation speeds in the urethane-rubber phantom and ethanol solutions can be estimated within errors of 3 and 2% in experiments.

Bone density and elasticity can be measured from propagation parameters of acoustic waves. In typical bone-density estimation, a propagation path of the ultrasonic wave has to be filled with mediums, whose acoustic properties are similar to tissues, to reduce attenuation by boundary reflection. Therefore, transducers are brought into contact with examined regions. In this study, non-contact bone-density estimation for a calcaneus using airborne ultrasonic waves has been proposed. When airborne ultrasonic waves pass through a heel, the received signals are attenuated more than -70 dB, which is even estimated from ideal flat boundaries of air, soft tissue, solid bone and cancellous bone. Therefore, pulse compression using M-sequence is employed to improve the S/N of received signals. When the order of M-sequence is n-th order, S/N improvement of received signals by cross correlation of the received signal and the reference signal, which is corresponding to the transmitted signal is √(2n - 1) times. The pass-through waves in a heel can be detected by amplification of 60 dB and S/N improvement of more than 45 dB in the experiments. However, the TOF of pass-through waves can be estimated because first-arrival pass-through waves are not clarified. For measurement of the propagation velocity of pass-through waves, wideband transducers or S/N improvement is required to clarify the first-arrival pass-through waves.

The acoustic measurement of vital information such as breathing and heartbeat in the standing position whilst the subject is wearing clothes is a difficult problem. In this paper, we present the basic experimental results to measure small movement of an object behind cloth. We measured acoustic characteristics of various types of cloth to obtain the transmission loss through cloth. To observe the relationship between measurement error and target speed under a low signal-to-noise ratio (SNR), we tried to measure the movement of an object behind cloth. The target was placed apart from the cloth to separate the target reflection from the cloth reflection. We found that a small movement of less than 6 mm/s could be observed using the M-sequence, moving target indicator (MTI) filter, and tracking phase difference, when the SNR was less than 0 dB. We also present the results of theoretical error analysis in the MTI filter and phase tracking for high-accuracy measurement. Characteristics of the systematic error were clarified.