Search

Humans are able to identify a large number of environmental sounds and categorise them according to high-level semantic categories, e.g. urban sounds or music. They are also capable of generalising from past experience to new sounds when applying these categories. In this paper we report on the creation of a data set that is structured according to the top-level of a taxonomy derived from human judgements and the design of an associated machine learning challenge, in which strong generalisation abilities are required to be successful. We introduce a baseline classification system, a deep convolutional network, which showed strong performance with an average accuracy on the evaluation data of 80.8%. The result is discussed in the light of two alternative explanations: An unlikely accidental category bias in the sound recordings or a more plausible true acoustic grounding of the high-level categories.

A core issue in speech perception and word recognition research is the nature of information perceivers use to identify spoken utterances across indexical variations in their phonetic details, such as talker and accent differences. Separately, a crucial question in audio-visual research is the nature of information perceivers use to recognize phonetic congruency between the audio and visual (talking face) signals that arise from speaking. We combined these issues in a study examining how differences between connected speech utterances (messages) versus between talkers and accents (messenger characteristics) contribute to recognition of crossmodal articulatory congruence between audio-only (AO) and video-only (VO) components of spoken utterances. Participants heard AO phrases in their native regional English accent or another English accent, and then saw two synchronous VO displays of point-light talking faces from which they had to select the one that corresponded to the audio target. The incorrect video in each pair was either the same or a different phrase as the audio target, produced by the same or a different talker, who spoke in either the same or a different English accent. Results indicate that cross-modal articulatory correspondence is more accurately and quickly detected for message content than for messenger details, suggesting that recognising the linguistic message is more fundamental than messenger features is to cross-modal detection of audio-visual articulatory congruency. Nonetheless, messenger characteristics, especially accent, affected performance to some degree, analogous to recent findings in AO speech research.

Social cues facilitate engagement between interaction participants, whether they be two (or more) humans or a human and an artificial agent such as a robot. Previous work specific to human–agent/robot interaction has demonstrated the efficacy of implemented social behaviours, such as eye-gaze or facial gestures, for demonstrating the illusion of engagement and positively impacting interaction with a human. We describe the implementation of THAMBS, The Thinking Head Attention Model and Behavioural System, which is used to model attention controlling how a virtual agent reacts to external audio and visual stimuli within the context of an interaction with a human user. We evaluate the efficacy of THAMBS for a virtual agent mounted on a robotic platform in a controlled experimental setting, and collect both task- and behavioural-performance variables, along with self-reported ratings of engagement. Our results show that human subjects noticeably engaged more often, and in more interesting ways, with the robotic agent when THAMBS was activated, indicating that even a rudimentary display of attention by the robot elicits significantly increased attention by the human. Back-channelling had less of an effect on user behaviour. THAMBS and back-channelling did not interact and neither had an effect on self-report ratings. Our results concerning THAMBS hold implications for the design of successful human–robot interactive behaviours.

Native speech perception is generally assumed to be highly efficient and accurate. Very little research has, however, directly examined the limitations of native perception, especially for contrasts that are only minimally differentiated acoustically and articulatorily. Here, we demonstrate that native speech perception may indeed be more difficult than is often assumed, where phonemes are highly similar, and we address the nature and extremes of consonant perception. We present two studies of native and non-native (English) perception of the acoustically and articulatorily similar four-way coronal stop contrast /t ? t ??/ (apicoalveolar, apico-retroflex, lamino-dental, lamino-alveopalatal) of Wubuy, an indigenous language of Australia. The results show that all listeners find contrasts involving /?/ easy to discriminate, but that, for both groups, contrasts involving /t ? t/? are much harder.Where the two groups differ, the results largely reflect native language (Wubuy vs English) attunement as predicted by the Perceptual Assimilation Model [1, 2, 3]. We also observe striking perceptual asymmetries in the native listeners' perception of contrasts involving the latter three stops, likely due to the differences in input frequency. Such asymmetries have not previously been observed in adults, and we propose a novel Natural Referent Consonant Hypothesis to account for the results.

Singing involves vocal production accompanied by a dynamic and meaningful use of facial expressions, which may serve as ancillary gestures that complement, disambiguate, or reinforce the acoustic signal. In this investigation, we examined the use of facial movements to communicate emotion, focusing on movements arising in three epochs: before vocalization (pre-production), during vocalization (production), and immediately after vocalization (post-production). The stimuli were recordings of seven vocalists' facial movements as they sang short (14 syllable) melodic phrases with the intention of communicating happiness, sadness, irritation, or no emotion. Facial movements were presented as point-light displays to 16 observers who judged the emotion conveyed. Experiment 1 revealed that the accuracy of emotional judgment varied with singer, emotion, and epoch. Accuracy was highest in the production epoch, however, happiness was well communicated in the pre-production epoch. In Experiment 2, observers judged point-light displays of exaggerated movements. The ratings suggested that the extent of facial and head movements was largely perceived as a gauge of emotional arousal. In Experiment 3, observers rated point-light displays of scrambled movements. Configural information was removed in these stimuli but velocity and acceleration were retained. Exaggerated scrambled movements were likely to be associated with happiness or irritation whereas unexaggerated scrambled movements were more likely to be identified as “neutral.” An analysis of singers' facial movements revealed systematic changes as a function of the emotional intentions of singers. The findings confirm the central role of facial expressions in vocal emotional communication, and highlight individual differences between singers in the amount and intelligibility of facial movements made before, during, and after vocalization.

In this study participants judged on the relationship between two interacting robots, one of them a mobile robot, the other one a stationary, robot arm-based artistic installation with a high flexibility in orienting its anthropomorphic face. The robots’ behaviour was either (1) weakly correlated through a loose tracking function, (2) independently random, or (3) independently random, but constrained to the same closely limited area. It was found that the true degree of coupling was reflected on average in the rating responses but that pseudo-random behaviour of one of the robots was judged less random if a relationship between the two robots was present. We argue that such robot-robot interaction experiments hold great value for social robotics as the interaction parameters are under complete control of the researchers.

Robots have diffidently started to invade human spaces, but are still limited to very rudimentary forms such as robot vacuum cleaners and various entertainment platforms. Dramatic changes with respect to the number of robots in homes and offices, however, can be foreseen for the near future as sensing, computing and associated technologies mature. Currently, it is not known how we humans will treat machine companions when they will be with us over prolonged periods of time and share our personal space. In this exploratory study we investigated whether participants would form a bond with a small, basic research robot in an adoption scenario whereby the robot's initial interaction abilities were upgraded in two steps. We were particularly interested in investigating whether any increases in attachment would be related to the 2 steps of progressively heightened technical sophistication of the robot over a prolonged (six month) period of time.

The present study investigates the acoustic differentiation of three coronal stops in the indigenous Australian language Wubuy. We test independent claims that only VC (vowel-into-consonant) transitions provide robust acoustic cues for retroflex as compared to alveolar and dental coronal stops, with no differentiating cues among these three coronal stops evident in CV (consonant-into-vowel) transitions. The four-way stop distinction /t, t̪, ʈ, c/ in Wubuy is contrastive word-initially (Heath 1984) and by implication utterance-initially, i.e., in CV-only contexts, which suggests that acoustic differentiation should be expected to occur in the CV transitions of this language, including in initial positions. Therefore, we examined both VC and CV formant transition information in the three target coronal stops across VCV (word-internal), V#CV (word-initial but utterance-medial) and ##CV (word- and utterance-initial), for /а/ vowel contexts, which provide the optimal environment for investigating formant transitions. Results confirm that these coronal contrasts are maintained in the CVs in this vowel context, and in all three positions. The patterns of acoustic differences across the three syllable contexts also provide some support for a systematic role of prosodic boundaries in influencing the degree of coronal stop differentiation evident in the vowel formant transitions.

The present study investigates the acoustic differentiation of three coronal stops in the indigenous Australian language Wubuy. We test independent claims that only VC (vowel-into-consonant) transitions provide robust acoustic cues for retroflex as compared to alveolar and dental coronal stops, with no differentiating cues among these three coronal stops evident in CV (consonant-into-vowel) transitions. The four-way stop distinction /t, t̪ , ʈ, c/ in Wubuy is contrastive word-initially (Heath 1984) and by implication utterance-initially, i.e., in CV-only contexts, which suggests that acoustic differentiation should be expected to occur in the CV transitions of this language, including in initial positions. Therefore, we examined both VC and CV formant transition information in the three target coronal stops across VCV (word-internal), V#CV (word-initial but utterance-medial) and ##CV (word- and utterance-initial), for /a / vowel contexts, which provide the optimal environment for investigating formant transitions. Results confirm that these coronal contrasts are maintained in the CVs in this vowel context, and in all three positions. The patterns of acoustic differences across the three syllable contexts also provide some support for a systematic role of prosodic boundaries in influencing the degree of coronal stop differentiation evident in the vowel formant transitions.

Although the increase in intelligibility of spoken language when watching the speaker’s face is well documented, the characteristics and the distribution of phonetic information over the frequency range of visual speech remain largely unknown. For this study face motion and tongue movements were measured simultaneously for three speakers of American English. The motion signals were subjected to a multiresolution analysis using spline wavelets and partial least squares regression was applied to estimate tongue movements from face motion. The amount of recovered variance was found to be small (below 30%) compared to previous studies and more pronounced in the lower frequencies.

We present a study aimed at investigating how novel signs emerge and spread through a community of interacting individuals. Ten triads of participants played a game in which players created novel signs in order to communicate with each other while constantly rotating between the role of interlocutor and that of observer. The main result of the study was that, for a majority of the triads, communicative success was not shared by the three dyads of players in a triad. This imbalance appears to be due to individual differences in game performance as well as to uncooperative behaviors. We suggest that both of these are magnified by the social dynamics induced by the role rotations in the game.

Research has indicated that native speech perception may be more difficult than is often assumed. Coronal stop contrastseries might fall in this category as they have been reported to be very difficult to distinguish even by native listeners, though this has not previously been systematically examined. It has been claimed that listeners rely on information in preceding vowels in order to correctly perceive these stops, making perception particularly difficult, if not impossible, when the stops are not preceded by a vowel. This paper presents two studies of the discrimination of multiple coronal stops by native speakers of Wubuy and provides strong evidence that native listeners are able to discriminate these consonants, even when they are not preceded by a vowel.

This paper tests the predictions of the vocabulary-tuning model of second language (L2) rephonologization in the domain of L2 segmental production. This model proposes a facilitating effect of adults’ L2 vocabulary expansion on L2 perception and production and suggests that early improvements in L2 segmental production may be positively associated with an expanding L2 vocabulary. The model was tested in a study of the L2 vowel intelligibility of adult Japanese learners of Australian English, who differed only in the size of their L2 vocabularies. The results support the predicted association between L2 vocabulary size and L2 vowel intelligibility and the prediction that early-phase L2 vocabulary expansion leads to improved L2 production.