Search

Contact events during manipulation tasks can be distinguished in two categories: intended and unintended. We investigated the categorization of contact events during an object placing task executed in virtual reality based on kinematic features measured during the movement segment previous to the contact. The experimental setup enabled us to generate unintended contacts by triggering unexpected interruptions of the placing movement. Experimental results indicate that the kinematic features enable the distinction of intended and unintended contacts independent of substantial variations of movement properties (amplitude, duration, velocity), unless the unintended contact occurs toward the end of the planned movement.

The evaluation of printing machines poses the problem of how distortions like streaks caused by the machine can be detected and assessed automatically. Although luminance variations in prints can be measured quite precisely, the measured functions bear little relevance for the lightness of streaks and other distortions of prints as perceived by human observers. First, the measurements sometimes indicate changes of luminance in regions which are perceived as homogeneous by humans. Second, the measured strength of a distortion correlates often weakly with its perceived strength, which is influenced by a variety of factors, like the shape of a streak’s luminance profile and the distribution of luminance variations in its spatial surround. We have used a model of human perception, based on fundamental neurophysiological and psychophysical properties of the visual system, in order to predict the perceptual strength of streaks (i.e. the distortion as perceived by a human observer) from the measured phys...

Many tasks involve the precise estimation of speed and position of moving objects, for instance to catch or avoid objects that cohabit in our environment. Many of these objects are characterised by signal representations in more than one modality, such as hearing and vision. The aim of this study was to investigate the extent to which the simultaneous presentation of auditory and visual signals enhances the estimation of motion speed and instantaneous position. Observers are asked to estimate the instant when a moving object arrives at a target spatial position by pressing a response button. This task requires observers to estimate the speed of the moving object and to calibrate the timing of their manual response such that it coincides with the true arrival time of the moving object. When both visual and auditory motion signals are available, the variability in estimating the arrival time of the moving object is significantly reduced compared to the variability in the unimodal conditions. This reduction in variability is consistent with optimal integration of the auditory and visual speed signals. The average bias in the estimated arrival times depends on the motion speed: for medium speeds (17deg/s) observers' subjective arrival times are earlier than the true arrival times; for high speeds (47deg/s) observers exhibit a (much smaller) bias in the other direction. This speed-dependency suggests that the bias is due to an error in estimating the motion speeds rather than an error in calibrating the timing of the motor response. Finally, in this temporal localization task, the bias and variability show similar patterns for motion defined by vision, audition or both.

It is well known that the detection thresholds for stationary auditory and visual signals are lower if the signals are presented bimodally rather than unimodally, provided the signals coincide in time and space. Recent work on auditory-visual motion detection suggests that the facilitation seen for stationary signals is not seen for motion signals. We investigate the conditions under which motion perception also benefits from the integration of auditory and visual signals. We show that the integration of cross-modal local motion signals that are matched in position and speed is consistent with thresholds predicted by a neural summation model. If the signals are presented in different hemi-fields, move in different directions, or both, then behavioural thresholds are predicted by a probability-summation model. We conclude that cross-modal signals have to be co-localised and co-incident for effective motion integration. We also argue that facilitation is only seen if the signals contain all localisation cues that would be produced by physical objects.

Linear filtering is a basic concept in neural models of early sensory information processing. In particular the visual system has been described to perform a wavelet-like multi-channel decomposition by a set of independent spatial-frequency selective filter mechanisms. Here we suggest that this principle of linear filtering deserves a critical re-evaluation. We propose that an optimal adaptation to natural scene statistics would require AND-like nonlinear interactions between the frequency-selective filter channels. We describe how this hypothesis can be tested by predicted violations of the principle of linearity that should be observable if cortical neurons would actually implement the proposed nonlinearities. We further explain why these effects might have been easily overlooked in earlier tests of the linearity of neurons in primary visual cortex.

We investigate the hypothesis that the main representation which underlies human navigation is not static and map-like, but rather is of an inherently sensorimotor nature, i.e. results from a combination of sensory features and motor actions. This is suggested by recent psychological and neurobiological results, and receives further support from an own study of human navigation in manipulated virtual reality environments. To investigate the presumed sensorimotor representation we design a hybrid architecture which integrates a bottom-up processing of sensorimotor features with a top-down reasoning that is based on the principle of maximum information gain. This architecture is implemented in an agent that operates in a VR environment and is able to use a minimum number of exploratory actions to orient itself within this environment.

Determining one's position within the environment is a basic feature of spatial behavior and spatial cognition. This task is of inherently sensorimotor nature in that it results from a combination of sensory features and motor actions, where the latter comprise exploratory movements to different positions in the environment. Biological agents achieve this in a robust and effortless fashion, which prompted us to investigate a bio-inspired architecture to study the localization process of an artificial agent which operates in virtual spatial environments. The spatial representation in this architecture is based on sensorimotor features that comprise sensory sensory features as well as motor actions. It is hierarchically organized and its structure can be learned in an unsupervised fashion by an appropriate clustering rule. In addition, the architecture has a temporal belief update mechanism which explicitly utilizes the statistical correlations of actions and locations. The architecture is hybrid in integrating bottom-up processing of sensorimotor features with topdown reasoning which is able to select optimal motor actions based on the principle of maximum information gain. The architecture operates on two sensorimotor levels, a macro-level, which controls the movements of the agent in space, and on a micro-level, which controls its eye movements. As a result, the virtual mobile agent is able to localize itself within an environment using a minimum number of exploratory actions.

The control of attention and of the associated saccadic fixations is to a large degree determined by the image content, that is, by salient local features in the image. The nature of this salient features and of the underlying strategy for their selection can be investigated within the context of natural scene statistics. For this, saccadic eye movements of human observers are recorded for a variety of natural and artificial test images, and the statistical properties of the fixated image regions are analyzed. The second-order statistics indicate that regions with higher spatial variance have a significantly higher probability to be fixated. Local contrast is therefore a salient feature. It was difficult to find additional differences in the local power spectra that can yield unequivocal information about a preference for specific local form properties. Differences that can be more easily interpreted as structural properties are derived from an investigation with higher-order statistics (bispectral density). The results indicate that nonredundant, intrinsically two-dimensional image features such as curved lines and edges, occlusions, isolated spots, and so on, play an important role in the saccadic selection process. Such features cannot be extracted by the classic spatial-frequency selective filter mechanisms but require nonlinear AND-like interactions between frequency components as possibly provided by end-stopping or hypercomplex properties related to the extraclassical receptive field.

Natural images contain considerable statistical redundancies beyond the level of second-order correlations. To identify the nature of these higher-order dependencies, we analyze the bispectra and trispectra of natural images. Our investigations reveal substantial statistical dependencies between those frequency components which are aligned to each other with respect to orientation. We argue that operators which are selective to local intrinsic dimensionality can optimally exploit such redundancies. We also show that the polyspectral structure we find for natural images helps to understand the hitherto unexplained superiority of orientation-selective filter decompositions over isotropic schemes like the Laplacian pyramid. However any essentially linear scheme can only partially exploit this higher-order redundancy. We therefore propose nonlinear i2D-selective operators which exhibit close resemblance to hypercomplex and end-stopped cells in the visual cortex. The function of these operators can be interpreted as a higher-order whitening of the input signal.

Based on an information theoretical approach, we investigate feature selection processes in saccadic object and scene analysis. Saccadic eye movements of human observers are recorded for a variety of natural and artificial test images. These experimental data are used for a statistical evaluation of the fixated image regions. Analysis of second-order statistics indicates that regions with higher spatial variance have a higher probability to be fixated, but no significant differences beyond these variance effects could be found at the level of power spectra. By contrast, an investigation with higher-order statistics, as reflected in the bispectral density, yielded clear structural differences between the image regions selected by saccadic eye movements as opposed to regions selected by a random process. These results indicate that nonredundant, intrinsically two-dimensional image features like curved lines and edges, occlusions, isolated spots, etc. play an important role in the saccadic selection process which must be integrated with top-down knowledge to fully predict object and scene analysis by human observers.

The perception of an image by a human observer is usually modeled as a parallel process in which all parts of the image are treated more or less equivalently, but in reality the analysis of scenes is a highly selective procedure, in which only a small subset of image locations is processed by the precise and efficient neural machinery of foveal vision. To understand the principles behind this selection of the 'informative' regions of images we have developed a hybrid system, which consists of a combination of a knowledge-based reasoning system wit a low-level preprocessing by linear and nonlinear neural operators. This hybrid system is intended as a first step towards a compete model of the sensorimotor system of saccadic scene analysis. In the analysis of a scene, the system calculates in each step which eye movement has to be made to reach a maximum of information about the scene. The possible information gain is calculated by means of a parallel strategy which is suitable for adaptive reasoning. The output of the system is a fixation sequence, and finally, a hypothesis about the scene.

The processing and representation of motion information is addressed from an integrated perspective comprising low- level signal processing properties as well as higher-level cognitive aspects. For the low-level processing of motion information we argue that a fundamental requirement is the existence of a spatio-temporal memory. Its key feature, the provision of an orthogonal relation between external time and its internal representation, is achieved by a mapping of temporal structure into a locally distributed activity distribution accessible in parallel by higher-level processing stages. This leads to a reinterpretation of the classical concept of `iconic memory' and resolves inconsistencies on ultra-short-time processing and visual masking. The spatial-temporal memory is further investigated by experiments on the perception of spatio-temporal patterns. Results on the direction discrimination of motion paths provide evidence that information about direction and location are not processed and represented independent of each other. This suggests a unified representation on an early level, in the sense that motion information is internally available in form of a spatio-temporal compound. For the higher-level representation we have developed a formal framework for the qualitative description of courses of motion that may occur with moving objects.

Sensory modalities are usually appropriately aligned in space: audition, vision, and proprioception each direct actions to the same spatial coordinates. Subjects wearing prism glasses that shift the visual input first miss the target in a pointing task, but quickly adapt to the new sensorimotor configuration. This adaptation may take place either in the visual or the proprioceptive pathway, ie, the internal visual representation is shifted back or the proprioceptive one is shifted towards the new configuration. Usually, the proprioceptive component is affected, probably due to the often observed dominance of vision over other modalities. This process is changed when auditory stimuli are presented during prism exposure inducing a shift of the visual representation contrary to the aforementioned results, maybe because a cortical mechanism performs statistical reliability estimation: Both audition and proprioception remain unaffected by prism exposure and therefore force vision to realign. We investigated the influence of sound-source-location on prism-adaptation by assessing the effects of displaced (accordingly to the prism offset), centered and unlocalized sound-sources. We discuss the influence of spatial properties on sensory calibration, its implications on the notion of motor action as the binding element between senses and its role in spatial adaptation processes.

Perception in natural environments is inseparably linked to motor action. In fact, we consider action an essential component of perceptual representation. But these representations are inherently difficult to investigate: Traditional experimental setups are limited by the lack of flexibility in manipulating spatial features. To overcome these problems, virtual reality (VR) experiments seem to be a feasible alternative, but these setups typically lack ecological realism due to the use of “unnatural” interface-devices (joystick). Thus, we propose an experimental apparatus which combines multisensory perception and action in an ecologically realistic way. The basis is a 10-foot hollow sphere (VirtuSphere) placed on a platform that allows free rotation. A subject inside can walk in any direction for any distance immersed into virtual environment. Both the rotation of the sphere and movement of the subject's head are tracked to process the subject's view within the VR-environment presented on a head-mounted display. Moreover, auditory features are dynamically processed taking greatest care of exact alignment of sound-sources and visual objects using ambisonic-encoded audio processed by a HRTF-filterbank. We present empirical data that confirm ecological realism of this setup and discuss its suitability for multi-sensory-motor research.

A coding scheme for image sequences is designed in analogy to human visual information processing. We propose a feature-specific vector quantization method applied to multi-channel representation of image sequences. The vector quantization combines the corresponding local/momentary amplitude coefficients of a set of three-dimensional analytic band-pass filters being selective for spatiotemporal frequency, orientation, direction and velocity. Motion compensation and decorrelation between successive frames is achieved implicitly by application of a non-rectangular subsampling to the 3D-bandpass outputs. The nonlinear combination of the outputs of filters which are selective for constantly moving one- dimensional (i.e. spatial elongated) image structures allows a classification of the local/momentary signal features with respect to their intrinsic dimensionality. Based on statistical investigations a natural hierarchy of signal features is provided. This is then used to construct an efficient encoding procedure. Thereby, the different sensitivity of the human vision to the various signal features can be easily incorporated. For a first example, all multi- dimensional vectors are mapped to constantly moving 1D-structures.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Intrinsic signal dimensionality, a property closely related to Gaussian curvature, is shown to be an important conceptual tool in multi-dimensional image processing for both biological and engineering sciences. Intrinsic dimensionality can reveal the relationship between recent theoretical developments in the definition of optic flow and the basic neurophysiological concept of 'end-stopping' of visual cortical cells. It is further shown how the concept may help to avoid certain problems typically arising from the common belief that an explicit computation of a flow field has to be the essential first step in the processing of spatio- temporal image sequences. Signals which cause difficulties in the computation of optic flow, mainly the discontinuities of the motion vector field, are shown to be detectable directly in the spatio-temporal input by evaluation of its three-dimensional curvature. The relevance of the suggested concept is supported by the fact that fast and efficient detection of such signals is of vital importance for ambulant observers in both the biological and the technical domain.

Image decomposition via even- and odd-symmetric, size and orientation selective band-pass filters, as suggested by the receptive field properties of visual cortical neurons, is well suited to image coding purposes. Interpretation of the even/odd filter outputs as a complex ("analytic") signal offers the alternative of a polar signal representation by a local amplitude and a local phase component. This is also indicated by our measurement of a rotation symmetric shape of the two-dimensional probability density function (pdf) of the even/odd filter outputs. Our investigations into the properties of such a representation show that it provides an interesting separation of the "amount of signal variation" (local amplitude) vs. the "type of signal variation" (local phase) . Furthermore an efficient vector quantization procedure can be applied to the two-dimensional amplitude/phase vector. This procedure divides the 2D signal space of the analytic signal into polar separable patches. Since phase quantization errors are more tolerable at small amplitude levels local phase is quantized dependent on the amplitude level. While typical pdf-optimized quantizers produce an increasingly higher amplitude resolution towards very small amplitudes, human vision allows the application of an appropriate threshold which leads to an "irrelevance zone" wherein obviously no phase information has to be coded. Using this coding scheme good image quality can be obtained with about 0.8 bit/pixel.© (1990) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We present an image coding scheme based on the properties of the early stages of the human visual system. The image signal is decomposed via even and odd symmetric, frequency and orientation selective band-pass filters in analogy to the quadrature phase simple cell pairs in the visual cortex. The resulting analytic signal is transformed into a local amplitude and local phase representation in order to achieve a better match to its signal statistics. Both intra filter dependencies of the analytic signal and inter filter dependencies between different orientation filters are exploited by a suitable vector quantization scheme. Inter orientation filter dependencies are demonstrated by means of a statistical evaluation of the multidimensional probability density function. The results can be seen as an empirical confirmation of the suitability of vector quantization in subband coding. Instead of generating a code book by use of an conventional design-algorithm, we suggest a feature specific partitioning of the multidimensional signal space matched to the properties of human vision. Using this coding scheme satisfactory image quality can be obtained with about 0.78 bit/pixel.

The representation of natural images by means of the analytical signal has certain principal advantages particular for textural components. A multi-channel model scheme is presented which enables to describe the distinctness of the textural impression and the difference between textures in analogy with the human observer. The role of phase and amplitude of the analytical signal in image representations is considered. As a result an effective image coding scheme is derived

The perception of an image by a human observer is usually modeled as a parallel process in which all parts of the image are treated more or less equivalently, but in reality the analysis of scenes is a highly selective procedure, in which only a small subset of image locations is processed by the precise and efficient neural machinery of foveal vision. To understand the principles behind this selection of the ‘‘informative’’ regions of images, we have developed a hybrid system that consists of a combination of a knowledgebased reasoning system with a low-level preprocessing by linear and nonlinear neural operators. This hybrid system is intended as a first step towards a complete model of the sensorimotor system of saccadic scene analysis. In the analysis of a scene, the system calculates in each step which eye movement has to be made to reach a maximum of information about the scene. The possible information gain is calculated by means of a parallel strategy which is suitable for adaptive reasoning. The output of the system is a fixation sequence, and finally, a hypothesis about the scene.

In this paper we develop a 4-dimensional representation for patterns based on image decompositions via orientation- and size-specific filters. By retaining image positional information, this encoding scheme reduces pattern rotations, translations, and scale changes to shifts in the filter outputs. The appropriate correlation processes for matching are discussed and the recognition system is illustrated by a number of examples.

A model for the perception of distortions in pictures is suggested. It consists of two main parts: an adaptive input stage realized as a ROG (Ratio of Gaussian) pyramid also suited for applications in image coding and computer vision, and a further decomposition by orientation selective filters including a saturating nonlinearity acting at each point of the filter outputs. The output values for each point of each filter are regarded as feature vector of the internal representation of the input picture. The difference between the internal representations of original and distorted picture is evaluated as norm of the difference vector. Due to local nonlinearities this operation explains periodic and aperiodic masking effects.