Search

Low- and high-pitched sounds are perceptually associated with low and high visuospatial elevations, respectively. The spatial properties of this association are not well understood. Here we report two experiments that investigated whether low and high tones can be used as spatial cues to upright for self-orientation and identified the spatial frame(s) of reference used in perceptually binding auditory pitch to visuospatial ‘up’ and ‘down’. In experiment 1, participants’ perceptual upright (PU) was measured while lying on their right side with and without high- and low-pitched sounds played through speakers above their left ear and below their right ear. The sounds were ineffective in moving the perceived upright from a direction intermediate between the body and gravity towards the direction indicated by the sounds. In experiment 2, we measured the biasing effects of ascending and descending tones played through headphones on ambiguous vertical or horizontal visual motion created by combining gratings drifting in opposite directions while participants either sat upright or laid on their right side. Ascending and descending tones biased the interpretation of ambiguous motion along both the gravitational vertical and the long-axis of the body with the strongest effect along the body axis. The combination of these two effects showed that axis of maximum effect of sound corresponded approximately to the direction of the perceptual upright, compatible with the idea that ‘high’ and ‘low’ sounds are defined along this axis.

The position of gaze (eye plus head position) relative to body is known to alter the perceived locations of sensory targets. This effect suggests that perceptual space is at least partially coded in a gaze-centered reference frame. However, the direction of the effects reported has not been consistent. Here, we investigate the cause of a discrepancy between reported directions of shift in tactile localization related to head position. We demonstrate that head eccentricity can cause errors in touch localization in either the same or opposite direction as the head is turned depending on the procedure used. When head position is held eccentric during both the presentation of a touch and the response, there is a shift in the direction opposite to the head. When the head is returned to center before reporting, the shift is in the same direction as head eccentricity. We rule out a number of possible explanations for the difference and conclude that when the head is moved between a touch and response the touch is coded in a predominantly gaze-centered reference frame, whereas when the head remains stationary a predominantly body-centered reference frame is used. The mechanism underlying these displacements in perceived location is proposed to involve an underestimated gaze signal. We propose a model demonstrating how this single neural error could cause localization errors in either direction depending on whether the gaze or body midline is used as a reference. This model may be useful in explaining gaze-related localization errors in other modalities.

Incorporating the fact that the senses are embodied is necessary for an organism to interpret sensory information. Before a unified perception of the world can be formed, sensory signals must be processed with reference to body representation. The various attributes of the body such as shape, proportion, posture, and movement can be both derived from the various sensory systems and can affect perception of the world (including the body itself). In this review we examine the relationships between sensory and motor information, body representations, and perceptions of the world and the body. We provide several examples of how the body affects perception (including but not limited to body perception). First we show that body orientation effects visual distance perception and object orientation. Also, visual-auditory crossmodal-correspondences depend on the orientation of the body: audio “high” frequencies correspond to a visual “up” defined by both gravity and body coordinates. Next, we show that perceived locations of touch is affected by the orientation of the head and eyes on the body, suggesting a visual component to coding body locations. Additionally, the reference-frame used for coding touch locations seems to depend on whether gaze is static or moved relative to the body during the tactile task. The perceived attributes of the body such as body size, affect tactile perception even at the level of detection thresholds and two-point discrimination. Next, long-range tactile masking provides clues to the posture of the body in a canonical body schema. Finally, ownership of seen body parts depends on the orientation and perspective of the body part in view. Together, all of these findings demonstrate how sensory and motor information, body representations, and perceptions (of the body and the world) are interdependent.

The perceived direction of up depends on visual, gravity, and body cues, each of which is given a weighting by the brain (Dyde et al., 2006). Little work has been done, however, to demonstrate whether sound might also be usable by the brain as a cue to up. Here we assess the possible contribution of sound to perceived orientation by adding a sound cue to gravity. The perceptual upright, the direction in which a character is most easily recognized, was assessed using the Oriented Character Recognition Test (OCHART). Subjects identified the character ‘p’ that was presented in various orientations (0–360 degrees rotation) as either a ‘p’ or ‘d’. The orientations were chosen by a QUEST adaptive staircase procedure and the mean of the points of subjective equality was taken as the perceptual upright. Subjects lay on their side and viewed a laptop screen through a shroud. Thus, body and gravity cues were orthogonal and swings of the perceptual upright towards or away from gravity could be measured. Loudspeakers were mounted above and below the lying subject (opposite the left and right ears) and sounds were presented synchronized to the appearance of the character on the screen. Changes in the direction of the PU were recorded depending on whether a sound was present or not. We conclude that sounds can contribute to the perception of upright.

Eccentric gaze systematically biases touch localization on the arm and waist. These perceptual errors suggest that touch location is at least partially coded in a visual reference frame. Here we investigated whether touches to non-visible parts of the body are also affected by gaze position. If so, can the direction of mislocalization tell us how they are laid out in the visual representation? To test this, an array of vibro-tactors was attached to either the lower back or the forehead. During trials, participants were guided to orient the position of their head (90° left, right or straight ahead for touches on the lower back) or head and eyes (combination of ±15° left, right or straight ahead head and eye positions for touches on the forehead) using LED fixation targets and a head mounted laser. Participants then re-oriented to straight ahead and reported perceived touch location on a visual scale using a mouse and computer screen. Similar to earlier experiments on the arm and waist, perceived touch location on the forehead and lower back was biased in the same direction as eccentric head and eye position. This is evidence that perceived touch location is at least partially coded in a visual reference frame even for parts of the body that are not typically seen.

We have previously reported that head position affects the perceived location of touch differently depending on the dynamics of the task the subject is involved in. When touch was delivered and responses were made with head rotated touch location shifted in the opposite direction to the head position, consistent with body-centered coding. When touch was delivered with head rotated but response was made with head centered touch shifted in the same direction as the head, consistent with gaze-centered coding. Here we tested whether moving the head in-between touch and response would modulate the effects of head position on touch location. Each trial consisted of three periods, in the first arrows and LEDs guided the subject to a randomly chosen head orientation (90° left, right, or center) and a vibration stimulus was delivered. Next, they were either guided to turn their head or to remain in the same location. In the final period they again were guided to turn or to remain in the same location before reporting the perceived location of the touch on a visual scale using a mouse and computer screen. Reported touch location was shifted in the opposite direction of head orientation during touch presentation regardless of the orientation during response or whether a movement was made before the response. The size of the effect was much reduced compared to our previous results. These results are consistent with touch location being coded in both a gaze centered and body centered reference frame during dynamic conditions.