Search

Our understanding of how visual systems detect, analyze and interpret visual stimuli has advanced greatly. However, the visual systems of all animals do much more; they enable visual behaviours. How well the visual system performs while interacting with the visual environment and how vision is used in the real world is far from fully understood, especially in humans. It has been suggested that comparison is the most primitive of psychophysical tasks. Thus, as a probe into these active visual behaviours, we use a same-different task: Are two physical 3D objects visually the same? This task is a fundamental cognitive ability. We pose this question to human subjects who are free to move about and examine two real objects in a physical 3D space. The experimental design is such that all behaviours are directed to viewpoint change. Without any training, our participants achieved a mean accuracy of 93.82%. No learning effect was observed on accuracy after many trials, but some effect was seen for response time, number of fixations and extent of head movement. Our probe task, even though easily executed at high-performance levels, uncovered a surprising variety of complex strategies for viewpoint control, suggesting that solutions were developed dynamically and deployed in a seemingly directed hypothesize-and-test manner tailored to the specific task. Subjects need not acquire task-specific knowledge; instead, they formulate effective solutions right from the outset, and as they engage in a series of attempts, those solutions progressively refine, becoming more efficient without compromising accuracy., (© 2023. The Author(s).)

The mechanisms involved in transforming early visual signals to curvature representations in V4 are unknown. We propose a hierarchical model that reveals V1/V2 encodings that are essential components for this transformation to the reported curvature representations in V4. Then, by relaxing the often-imposed prior of a single Gaussian, V4 shape selectivity is learned in the last layer of the hierarchy from Macaque V4 responses. We found that V4 cells integrate multiple shape parts from the full spatial extent of their receptive fields with similar excitatory and inhibitory contributions. Our results uncover new details in existing data about shape selectivity in V4 neurons that with additional experiments can enhance our understanding of processing in this area. Accordingly, we propose designs for a stimulus set that allow removing shape parts without disturbing the curvature signal to isolate part contributions to V4 responses. SIGNIFICANCE STATEMENT Selectivity to convex and concave shape parts in V4 neurons has been repeatedly reported. Nonetheless, the mechanisms that yield such selectivities in the ventral stream remain unknown. We propose a hierarchical computational model that incorporates findings of the various visual areas involved in shape processing and suggest mechanisms that transform the shape signal from low-level features to convex/concave part representations. Learning shape selectivity from Macaque V4 responses in the final processing stage in our model, we found that V4 neurons integrate shape parts from the full spatial extent of their receptive field with both facilitatory and inhibitory contributions. These results reveal hidden information in existing V4 data that with additional experiments can enhance our understanding of processing in V4., (Copyright © 2023 Mehrani and Tsotsos.)

Shifting the focus of attention without moving the eyes poses challenges for signal coding in visual cortex in terms of spatial resolution, signal routing, and cross-talk. Little is known how these problems are solved during focus shifts. Here, we analyze the spatiotemporal dynamic of neuromagnetic activity in human visual cortex as a function of the size and number of focus shifts in visual search. We find that large shifts elicit activity modulations progressing from highest (IT) through mid-level (V4) to lowest hierarchical levels (V1). Smaller shifts cause those modulations to start at lower levels in the hierarchy. Successive shifts involve repeated backward progressions through the hierarchy. We conclude that covert focus shifts arise from a cortical coarse-to-fine process progressing from retinotopic areas with larger toward areas with smaller receptive fields. This process localizes the target and increases the spatial resolution of selection, which resolves the above issues of cortical coding.

Background: Feature-based attention prioritizes the processing of the attended feature while strongly suppressing the processing of nearby ones. This creates a non-linearity or "attentional suppressive surround" predicted by the Selective Tuning model of visual attention. However, previously reported effects of feature-based attention on neuronal responses are linear, e.g., feature-similarity gain. Here, we investigated this apparent contradiction by neurophysiological and psychophysical approaches., Results: Responses of motion direction-selective neurons in area MT/MST of monkeys were recorded during a motion task. When attention was allocated to a stimulus moving in the neurons' preferred direction, response tuning curves showed its minimum for directions 60-90° away from the preferred direction, an attentional suppressive surround. This effect was modeled via the interaction of two Gaussian fields representing excitatory narrowly tuned and inhibitory widely tuned inputs into a neuron, with feature-based attention predominantly increasing the gain of inhibitory inputs. We further showed using a motion repulsion paradigm in humans that feature-based attention produces a similar non-linearity on motion discrimination performance., Conclusions: Our results link the gain modulation of neuronal inputs and tuning curves examined through the feature-similarity gain lens to the attentional impact on neural population responses predicted by the Selective Tuning model, providing a unified framework for the documented effects of feature-based attention on neuronal responses and behavior., (© 2022. The Author(s).)

When we study the human ability to attend, what exactly do we seek to understand? It is not clear what the answer might be to this question. There is still so much to know, while acknowledging the tremendous progress of past decades of research. It is as if each new study adds a tile to the mosaic that, when viewed from a distance, we hope will reveal the big picture of attention. However, there is no map as to how each tile might be placed nor any guide as to what the overall picture might be. It is like digging up bits of mosaic tile at an ancient archeological site with no key as to where to look and then not only having to decide which picture it belongs to but also where exactly in that puzzle it should be placed. I argue that, although the unearthing of puzzle pieces is very important, so is their placement, but this seems much less emphasized. We have mostly unearthed a treasure trove of puzzle pieces but they are all waiting for cleaning and reassembly. It is an activity that is scientifically far riskier, but with great risk comes a greater reward. Here, I will look into two areas of broad agreement, specifically regarding visual attention, and dig deeper into their more nuanced meanings, in the hope of sketching a starting point for the guide to the attention mosaic. The goal is to situate visual attention as a purely computational problem and not as a data explanation task; it may become easier to place the puzzle pieces once you understand why they exist in the first place.

Rubin's face-vase illusion demonstrates how one can switch back and forth between two different interpretations depending on how the figure outlines are assigned. In the primate visual system, assigning ownership along figure borders is encoded by neurons called the border ownership (BO) cells. Studies show that the responses of these neurons not only depend on the local features within their receptive fields, but also on contextual information. Despite two decades of studies on BO neurons, the ownership assignment mechanism in the brain is still unknown. Here, we propose a hierarchical recurrent model grounded on the hypothesis that neurons in the dorsal stream provide the context required for ownership assignment. Our proposed model incorporates early recurrence from the dorsal pathway as well as lateral modulations within the ventral stream. While dorsal modulations initiate the response difference to figure on either side of the border, lateral modulations enhance the difference. We found responses of our dorsally-modulated BO cells, similar to their biological counterparts, are invariant to size, position and solid/outlined figures. Moreover, our model BO cells exhibit comparable levels of reliability in the ownership signal to biological BO neurons. We found dorsal modulations result in high levels of accuracy and robustness for BO assignments in complex scenes compared to previous models based on ventral feedback. Finally, our experiments with illusory contours suggest that BO encoding could explain the perception of such contours in higher processing stages in the brain., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

The study of attentional processing in vision has a long and deep history. Recently, several papers have presented insightful perspectives into how the coordination of multiple attentional functions in the brain might occur. These begin with experimental observations and the authors propose structures, processes, and computations that might explain those observations. Here, we consider a perspective that past works have not, as a complementary approach to the experimentally-grounded ones. We approach the same problem as past authors but from the other end of the computational spectrum, from the problem nature, as Marr's Computational Level would prescribe. What problem must the brain solve when orchestrating attentional processes in order to successfully complete one of the myriad possible visuospatial tasks at which we as humans excel? The hope, of course, is for the approaches to eventually meet and thus form a complete theory, but this is likely not soon. We make the first steps towards this by addressing the necessity of attentional control, examining the breadth and computational difficulty of the visuospatial and attentional tasks seen in human behavior, and suggesting a sketch of how attentional control might arise in the brain. The key conclusions of this paper are that an executive controller is necessary for human attentional function in vision, and that there is a 'first principles' computational approach to its understanding that is complementary to the previous approaches that focus on modelling or learning from experimental observations directly., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

There is still much to understand about the brain's colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain's colour processing pathway.

Eye movements during visual search change with prior experience for search stimuli. Previous studies measured these gaze effects shortly after initial viewing, typically during free viewing; it remains open whether the effects are preserved across long delays and for goal-directed search, and which memory system guides gaze. In Experiment 1, we analyzed eye movements of healthy adults viewing novel and repeated scenes while searching for a scene-embedded target. The task was performed across different time points to examine the repetition effects in long-term memory, and memory types were grouped based on explicit recall of targets. In Experiment 2, an amnesic person with bilateral extended hippocampal damage and the age-matched control group performed the same task with shorter intervals to determine whether or not the repetition effects depend on hippocampal function. When healthy adults explicitly remembered repeated target-scene pairs, search time and fixation duration decreased, and gaze was directed closer to the target region, than when they forgot targets. These effects were seen even after a one-month delay from their initial viewing, suggesting the effects are associated with long-term, explicit memory. Saccadic amplitude was not strongly modulated by scene repetition or explicit recall of targets. The amnesic person did not show explicit recall or implicit repetition effects, whereas his control group showed similar patterns to those seen in Experiment 1. The results reveal several aspects of gaze control that are influenced by long-term memory. The dependence of gaze effects on medial temporal lobe integrity support a role for this region in predictive gaze control.

Given an existing trained neural network, it is often desirable to learn new capabilities without hindering performance of those already learned. Existing approaches either learn sub-optimal solutions, require joint training, or incur a substantial increment in the number of parameters for each added domain, typically as many as the original network. We propose a method called Deep Adaptation Modules (DAM) that constrains newly learned filters to be linear combinations of existing ones. DAMs precisely preserve performance on the original domain, require a fraction (typically 13 percent, dependent on network architecture) of the number of parameters compared to standard fine-tuning procedures and converge in less cycles of training to a comparable or better level of performance. When coupled with standard network quantization techniques, we further reduce the parameter cost to around 3 percent of the original with negligible or no loss in accuracy. The learned architecture can be controlled to switch between various learned representations, enabling a single network to solve a task from multiple different domains. We conduct extensive experiments showing the effectiveness of our method on a range of image classification tasks and explore different aspects of its behavior.

The current dominant visual processing paradigm in both human and machine research is the feedforward, layered hierarchy of neural-like processing elements. Within this paradigm, visual saliency is seen by many to have a specific role, namely that of early selection. Early selection is thought to enable very fast visual performance by limiting processing to only the most salient candidate portions of an image. This strategy has led to a plethora of saliency algorithms that have indeed improved processing time efficiency in machine algorithms, which in turn have strengthened the suggestion that human vision also employs a similar early selection strategy. However, at least one set of critical tests of this idea has never been performed with respect to the role of early selection in human vision. How would the best of the current saliency models perform on the stimuli used by experimentalists who first provided evidence for this visual processing paradigm? Would the algorithms really provide correct candidate sub-images to enable fast categorization on those same images? Do humans really need this early selection for their impressive performance? Here, we report on a new series of tests of these questions whose results suggest that it is quite unlikely that such an early selection process has any role in human rapid visual categorization., Competing Interests: The authors have declared that no competing interests exist.

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing., Competing Interests: The authors have declared that no competing interests exist.

In adulthood, research has demonstrated that surrounding the spatial location of attentional focus is a suppressive field, resulting from top-down attention promoting the processing of relevant stimuli and inhibiting surrounding distractors (e.g., Hopf et al., 2006). It is not fully known, however, how this phenomenon manifests during development. This is an important question since attention processes are likely even more critical in development because of their potential impact on learning and day-to-day activities. The current study examined whether spatial suppression surrounding the focus of visual attention, a predicted by-product of top-down attentional modulation, is observed in development. A wide age range separated in six incremental age levels was included, allowing for a detailed examination of potential differences in the effect of attention on visual processing across development. Participants between 12 and 27 years of age exhibited spatial suppression surrounding their focus of visual attention. Their accuracy increased as a function of the separation distance between a spatially cued (and attended) target and a second target, suggesting that a ring of suppression surrounded the attended target. Attentional surround suppression was not observed in 8- to 11-years-olds, even with a longer spatial cue presentation time, demonstrating that the lack of the effect at these ages is not due to slowed attentional feedback processes. Our findings demonstrate that top-down attentional processes exhibit functional maturity beginning around 12 years of age with continuing maturation of their expression until 17, which likely impacts education and the diagnosis of visual and cognitive clinical pathologies.

The human capability to attend has been both considered as easy and as impossible to understand by philosophers and scientists through the centuries. Much has been written by brain, cognitive, and philosophical scientists trying to explain attention as it applies to sensory and reasoning processes, let alone consciousness. It has been only in the last few decades that computational scientists have entered the picture adding a new language with which to express attentional behavior and function. This new perspective has produced some progress to the centuries-old goal, but there is still far to go. Although a central belief in many scientific disciplines has been to seek a unifying explanatory principle for natural observations, it may be that we need to put this aside as it applies to attention and accept the fact that attention is really an integrated set of mechanisms, too messy to cleanly and parsimoniously express with a single principle. These mechanisms are claimed to be critical to enable functional generalization of brain processes and thus an integrative perspective is important. Here we present first steps towards a theoretical and algorithmic view on how the many different attentional mechanisms may be deployed, coordinated, synchronized, and effectively utilized. A hierarchy of dynamically defined closed-loop control processes is proposed, each with its own optimization objective, which is extensible to multiple layers. Although mostly speculative, simulation and experimental work support important components.

The Selective Tuning model of visual attention (Tsotsos, 1990) has proposed that the focus of attention is surrounded by an inhibitory zone, eliciting a center-surround attentional distribution. This attentional suppressive surround inhibits irrelevant information which is located close to attended information in physical space (e.g., Cutzu and Tsotsos, 2003; Hopf et al., 2010) or in feature space (e.g., Tombu and Tsotsos, 2008; Störmer and Alvarez, 2014; Bartsch et al., 2017). In Experiment 1, we investigate the interaction between location-based and feature-based surround suppression and hypothesize that the attentional surround suppression would be maximized when spatially adjacent stimuli are also represented closely within a feature map. Our results demonstrate that perceptual discrimination is worst when two similar orientations are presented in proximity to each other, suggesting the interplay of the two surround suppression mechanisms. The Selective Tuning model also predicts that the size of the attentional suppressive surround is determined by the receptive field size of the neuron which optimally processes the attended information. The receptive field size of the processing neurons is tightly associated with stimulus size and eccentricity. Therefore, Experiment 2 tested the hypothesis that the size of the attentional suppressive surround would become larger as stimulus size and eccentricity increase, corresponding to an increase in the neuron's receptive field size. We show that stimulus eccentricity but not stimulus size modulates the size of the attentional suppressive surround. These results are consistent for both low- and high-level features (e.g., orientation and human faces). Overall, the present study supports the existence of the attentional suppressive surround and reveals new properties of this selection mechanism.

It is almost universal to regard attention as the facility that permits an agent, human or machine, to give priority processing resources to relevant stimuli while ignoring the irrelevant. The reality of how this might manifest itself throughout all the forms of perceptual and cognitive processes possessed by humans, however, is not as clear. Here, we examine this reality with a broad perspective in order to highlight the myriad ways that attentional processes impact both perception and cognition. The paper concludes by showing two real-world problems that exhibit sufficient complexity to illustrate the ways in which attention and cognition connect. These then point to new avenues of research that might illuminate the overall cognitive architecture of spatial cognition.

Attention modulates neural selectivity and optimizes the allocation of cortical resources during visual tasks. A large number of experimental studies in primates and humans provide ample evidence. As an underlying principle of visual attention, some theoretical models suggested the existence of a gain element that enhances contrast of the attended stimuli. In contrast, the Selective Tuning model of attention (ST) proposes an attentional mechanism based on suppression of irrelevant signals. In this paper, we present an updated characterization of the ST-neuron proposed by the Selective Tuning model, and suggest that the inclusion of adaptation currents (Ih) to ST-neurons may explain the temporal profiles of the firing rates recorded in single V4 cells during attentional tasks. Furthermore, using the model we show that the interaction between stimulus-selectivity of a neuron and attention shapes the profile of the firing rate, and is enough to explain its fast modulation and other discontinuities observed, when the neuron responds to a sudden switch of stimulus, or when one stimulus is added to another during a visual task.

Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception. A multitude of ideas and methods for how to accomplish this have already appeared in the past, their broader utility perhaps impeded by insufficient computational power or costly hardware. The history of these ideas, perhaps selective due to our perspectives, is presented with the goal of organizing the past literature and highlighting the seminal contributions. We argue that those contributions are as relevant today as they were decades ago and, with the state of modern computational tools, are poised to find new life in the robotic perception systems of the next decade., (© The Author(s) 2017.)

Attention can facilitate the selection of elementary object features such as color, orientation, or motion. This is referred to as feature-based attention and it is commonly attributed to a modulation of the gain and tuning of feature-selective units in visual cortex. Although gain mechanisms are well characterized, little is known about the cortical processes underlying the sharpening of feature selectivity. Here, we show with high-resolution magnetoencephalography in human observers (men and women) that sharpened selectivity for a particular color arises from feedback processing in the human visual cortex hierarchy. To assess color selectivity, we analyze the response to a color probe that varies in color distance from an attended color target. We find that attention causes an initial gain enhancement in anterior ventral extrastriate cortex that is coarsely selective for the target color and transitions within ∼100 ms into a sharper tuned profile in more posterior ventral occipital cortex. We conclude that attention sharpens selectivity over time by attenuating the response at lower levels of the cortical hierarchy to color values neighboring the target in color space. These observations support computational models proposing that attention tunes feature selectivity in visual cortex through backward-propagating attenuation of units less tuned to the target. SIGNIFICANCE STATEMENT Whether searching for your car, a particular item of clothing, or just obeying traffic lights, in everyday life, we must select items based on color. But how does attention allow us to select a specific color? Here, we use high spatiotemporal resolution neuromagnetic recordings to examine how color selectivity emerges in the human brain. We find that color selectivity evolves as a coarse to fine process from higher to lower levels within the visual cortex hierarchy. Our observations support computational models proposing that feature selectivity increases over time by attenuating the responses of less-selective cells in lower-level brain areas. These data emphasize that color perception involves multiple areas across a hierarchy of regions, interacting with each other in a complex, recursive manner., (Copyright © 2017 the authors 0270-6474/17/3710346-12$15.00/0.)

Much has been written about how the biological brain might represent and process visual information, and how this might inspire and inform machine vision systems. Indeed, tremendous progress has been made, and especially during the last decade in the latter area. However, a key question seems too often, if not mostly, be ignored. This question is simply: do proposed solutions scale with the reality of the brain's resources? This scaling question applies equally to brain and to machine solutions. A number of papers have examined the inherent computational difficulty of visual information processing using theoretical and empirical methods. The main goal of this activity had three components: to understand the deep nature of the computational problem of visual information processing; to discover how well the computational difficulty of vision matches to the fixed resources of biological seeing systems; and, to abstract from the matching exercise the key principles that lead to the observed characteristics of biological visual performance. This set of components was termed complexity level analysis in Tsotsos (1987) and was proposed as an important complement to Marr's three levels of analysis. This paper revisits that work with the advantage that decades of hindsight can provide.

In the past decade, a large number of computational models of visual saliency have been proposed. Recently a number of comprehensive benchmark studies have been presented, with the goal of assessing the performance landscape of saliency models under varying conditions. This has been accomplished by considering fixation data, annotated image regions, and stimulus patterns inspired by psychophysics. In this paper, we present a high-level examination of challenges in computational modeling of visual saliency, with a heavy emphasis on human vision and neural computation. This includes careful assessment of different metrics for performance of visual saliency models, and identification of remaining difficulties in assessing model performance. We also consider the importance of a number of issues relevant to all saliency models including scale-space, the impact of border effects, and spatial or central bias. Additionally, we consider the biological plausibility of models in stepping away from exemplar input patterns towards a set of more general theoretical principles consistent with behavioral experiments. As a whole, this presentation establishes important obstacles that remain in visual saliency modeling, in addition to identifying a number of important avenues for further investigation., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

Scores of visual attention models have been developed over the past several decades of research. Differences in implementation, assumptions, and evaluations have made comparison of these models very difficult. Taxonomies have been constructed in an attempt at the organization and classification of models, but are not sufficient at quantifying which classes of models are most capable of explaining available data. At the same time, a multitude of physiological and behavioral findings have been published, measuring various aspects of human and non-human primate visual attention. All of these elements highlight the need to integrate the computational models with the data by (1) operationalizing the definitions of visual attention tasks and (2) designing benchmark datasets to measure success on specific tasks, under these definitions. In this paper, we provide some examples of operationalizing and benchmarking different visual attention tasks, along with the relevant design considerations., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

What are the computational tasks that an executive controller for visual attention must solve? This question is posed in the context of the Selective Tuning model of attention. The range of required computations go beyond top-down bias signals or region-of-interest determinations, and must deal with overt and covert fixations, process timing and synchronization, information routing, memory, matching control to task, spatial localization, priming, and coordination of bottom-up with top-down information. During task execution, results must be monitored to ensure the expected results. This description includes the kinds of elements that are common in the control of any kind of complex machine or system. We seek a mechanistic integration of the above, in other words, algorithms that accomplish control. Such algorithms operate on representations, transforming a representation of one kind into another, which then forms the input to yet another algorithm. Cognitive Programs (CPs) are hypothesized to capture exactly such representational transformations via stepwise sequences of operations. CPs, an updated and modernized offspring of Ullman's Visual Routines, impose an algorithmic structure to the set of attentional functions and play a role in the overall shaping of attentional modulation of the visual system so that it provides its best performance. This requires that we consider the visual system as a dynamic, yet general-purpose processor tuned to the task and input of the moment. This differs dramatically from the almost universal cognitive and computational views, which regard vision as a passively observing module to which simple questions about percepts can be posed, regardless of task. Differing from Visual Routines, CPs explicitly involve the critical elements of Visual Task Executive (vTE), Visual Attention Executive (vAE), and Visual Working Memory (vWM). Cognitive Programs provide the software that directs the actions of the Selective Tuning model of visual attention.

Our ever-growing knowledge about the human brain and human behavior is opening doors to increasingly impressive technological achievements. This neurobiological inspiration has a significant history and involves almost equal parts neuroscience, computation and art. With a focus on the sense of vision, this essay presents a selective and highly condensed snapshot of the history of how neurobiology has inspired technological developments, pointing the way to where new inspirations may lead., (Copyright © 2014 Elsevier Ltd. All rights reserved.)

Various models of the neural mechanisms of attentional modulation in the visual cortex have been proposed. In general, these models assume that an 'attention' parameter is provided separately. Its value as well as the selection of neuron(s) to which it applies are assumed, but its source and the selection mechanism are unspecified. Here we show how the Selective Tuning model of visual attention can account for the modulation of the firing rate at the single neuron level, and for the temporal pattern of attentional modulations in the visual cortex, in a self-contained formulation that simultaneously determines the stimulus elements to be attended while modulating the relevant neural processes.

That shape is important for perception has been known for almost a thousand years (thanks to Alhazen in 1083) and has been a subject of study ever since by scientists and phylosophers (such as Descartes, Helmholtz or the Gestalt psychologists). Shapes are important object descriptors. If there was any remote doubt regarding the importance of shape, recent experiments have shown that intermediate areas of primate visual cortex such as V2, V4 and TEO are involved in analyzing shape features such as corners and curvatures. The primate brain appears to perform a wide variety of complex tasks by means of simple operations. These operations are applied across several layers of neurons, representing increasingly complex, abstract intermediate processing stages. Recently, new models have attempted to emulate the human visual system. However, the role of intermediate representations in the visual cortex and their importance have not been adequately studied in computational modeling.This paper proposes a model of shape-selective neurons whose shape-selectivity is achieved through intermediate layers of visual representation not previously fully explored. We hypothesize that hypercomplex--also known as endstopped--neurons play a critical role to achieve shape selectivity and show how shape-selective neurons may be modeled by integrating endstopping and curvature computations. This model--a representational and computational system for the detection of 2-dimensional object silhouettes that we term 2DSIL--provides a highly accurate fit with neural data and replicates responses from neurons in area V4 with an average of 83% accuracy. We successfully test a biologically plausible hypothesis on how to connect early representations based on Gabor or Difference of Gaussian filters and later representations closer to object categories without the need of a learning phase as in most recent models.

Most current algorithm evaluation protocols use large image databases, but give little consideration to imaging characteristics used to create the data sets. This paper evaluates the effects of camera shutter speed and voltage gain under simultaneous changes in illumination and demonstrates significant differences in the sensitivities of popular vision algorithms under variable illumination, shutter speed, and gain. These results show that offline data sets used to evaluate vision algorithms typically suffer from a significant sensor specific bias which can make many of the experimental methodologies used to evaluate vision algorithms unable to provide results that generalize in less controlled environments. We show that for typical indoor scenes, the different saturation levels of the color filters are easily reached, leading to the occurrence of localized saturation which is not exclusively based on the scene radiance but on the spectral density of individual colors present in the scene. Even under constant illumination, foreshortening effects due to surface orientation can affect feature detection and saliency. Finally, we demonstrate that active and purposive control of the shutter speed and gain can lead to significantly more reliable feature detection under varying illumination and nonconstant viewpoints.

In visual search experiments there exist a variety of experimental paradigms in which a symmetric set of experimental conditions yields asymmetric corresponding task performance. There are a variety of examples of this that currently lack a satisfactory explanation. In this paper, we demonstrate that distinct classes of asymmetries may be explained by virtue of a few simple conditions that are consistent with current thinking surrounding computational modeling of visual search and coding in the primate brain. This includes a detailed look at the role that stimulus familiarity plays in the determination of search performance. Overall, we demonstrate that all of these asymmetries have a common origin, namely, they are a consequence of the encoding that appears in the visual cortex. The analysis associated with these cases yields insight into the problem of visual search in general and predictions of novel search asymmetries.

Visual attention biases relevant processing in the visual system by amplifying relevant or attenuating irrelevant sensory input. A potential signature of the latter operation, referred to as surround attenuation, has recently been identified in the electromagnetic brain response of human observers performing visual search. It was found that a zone of attenuated cortical excitability surrounds the target when the search required increased spatial resolution for item discrimination. Here we address the obvious hypothesis that surround attenuation serves distractor suppression in the vicinity of the target where interference from irrelevant search items is maximal. To test this hypothesis, surround attenuation was assessed under conditions when the target was presented in isolation versus when it was surrounded by distractors. Surprisingly, substantial and indistinguishable surround attenuation was seen under both conditions, indicating that it reflects an attentional operation independent of the presence of distractors. Adding distractors in the target's surround, however, increased the amplitude of the N2pc--an evoked response known to index distractor competition in visual search. Moreover, adding distractors led to a topographical change of source activity underlying the N2pc toward earlier extrastriate areas. In contrast, the topography of reduced source activity due to surround attenuation remained unaltered with and without distractors in the target's surround. We conclude that surround attenuation is not a direct consequence of the attenuation of distractors in visual search and that it dissociates from attentional operations reflected by the N2pc. A theoretical framework is proposed that links both operations in a common model of top-down attentional selection in visual cortex.

The spatial focus of attention has been suggested to resemble a spotlight, a zoom-lens, a simple gradient, or even a more complex center-surround profile. Here we review evidence from neuromagnetic recordings indicating that the spatial profile is not fixed but depends on the particular perceptual demands of the attention task. We show that visual search requiring spatial scrutiny for target discrimination produces a zone of neural attenuation in the target's immediate surround, whereas search permitting target discrimination without spatial scrutiny is associated with a simple gradient. We provide new evidence indicating that increasing the demands on target discrimination without changing the spatial scale of discrimination does not influence surround attenuation, and that surround attenuation is also not influenced by the type of features involved in forward processing, that is whether the target location is defined by color or luminance contrast in visual search. An assessment of the time-course of attentional selection reveals that, when present, surround attenuation onsets with a substantial delay relative to the initial feed-forward sweep of processing in the visual system. The reported observations together suggest that the more complex center-surround profile arises as a consequence of top-down attentional selection in the visual system. The reviewed neuromagnetic evidence is discussed with respect to key notions of the Selective Tuning model of visual attention for which strong support is provided., (Copyright 2010 Elsevier Ltd. All rights reserved.)

As we move, the projection of moving objects on our retinas generates an array of velocity vectors known as optic flow. One class of optic flow is spiral motion, defined by the angle between a local vector direction and the direction of the steepest increase in local speed. By discriminating among such angles, an organism could discern between different flow patterns and effectively interact with the environment. In primates, spiral-selective neurons in medial superior temporal area are thought to provide the substrate for this ability. We found that these cells show higher discrimination thresholds than found behaviorally in humans, suggesting that when discriminating spiral motions the brain integrates information across many of these neurons to achieve its high perceptual performance.

We recently demonstrated with magnetoencephalographic recordings in human observers that the focus of attention in visual search has a spatial profile consisting of a center enhancement surrounded by a narrow zone of sensory attenuation. Here, we report new data from 2 experiments providing insights into the cortical processes that cause the surround attenuation. We show that surround suppression appears in search tasks that require spatial scrutiny, that is the precise binding of search-relevant features at the target's location but not in tasks that permit target discrimination without precise localization. Furthermore, we demonstrate that surround attenuation is linked with a stronger recurrent activity modulation in early visual cortex. Finally, we show that surround suppression appears with a delay (more than 175 ms) that is beyond the time course of the initial feedforward sweep of processing in the visual system. These observations together indicate that the suppressive surround is associated with recurrent processing and binding in the visual cortex.

A proposal for saliency computation within the visual cortex is put forth based on the premise that localized saliency computation serves to maximize information sampled from one's environment. The model is built entirely on computational constraints but nevertheless results in an architecture with cells and connectivity reminiscent of that appearing in the visual cortex. It is demonstrated that a variety of visual search behaviors appear as emergent properties of the model and therefore basic principles of coding and information transmission. Experimental results demonstrate greater efficacy in predicting fixation patterns across two different data sets as compared with competing models.

An object may afford a number of different actions. In this article, we show that an attentional mechanism inhibits competing motor programs that could elicit erroneous actions. Participants made a speeded key press to categorize the second of two successively presented door handles that were rotated at varying orientations relative to one another. Their responding hand was compatible or incompatible with the graspable part of the door handles (rightward or leftward facing). Compatible responses were faster than incompatible responses if the two handles shared an identical orientation, but they were slower if the two handles were aligned at slightly dissimilar orientations. Such suppressive surround effects are hallmarks of attentional processing in the visual domain, but they have never been observed behaviorally in the motor domain. This finding delineates a common mechanism involved in two of the most important functions of the brain: processing sensory data and preparing actions based on that information.

Many think that visual attention needs an executive to allocate resources. Although the cortex exhibits substantial plasticity, dynamic allocation of neurons seems outside its capability. Suppose instead that the visual processing architecture is fixed, but can be 'tuned' dynamically to task requirements: the only remaining resource that can be allocated is time. How can this fixed, yet tunable, structure be used over periods of time longer than one feed-forward pass? With the goal of developing a computational theory and model of vision and attention that has both biological predictive power as well as utility for computer vision, this paper proposes that by using multiple passes of the visual processing hierarchy, both bottom-up and top-down, and using task information to tune the processing prior to each pass, we can explain the different recognition behaviors that human vision exhibits. By examining in detail the basic computational infrastructure provided by the Selective Tuning model and using its functionality, four different binding processes - Convergence Binding and Partial, Full and Iterative Recurrence Binding - are introduced and tied to specific recognition tasks and their time course. The key is a provable method to trace neural activations through multiple representations from higher order levels of the visual processing network down to the early levels.

We present a framework for the analysis of short axis cardiac MRI, using statistical models of shape and appearance. The framework integrates temporal and structural constraints and avoids common optimization problems inherent in such high dimensional models. The first contribution is the introduction of an algorithm for fitting 3D active appearance models (AAMs) on short axis cardiac MRI. We observe a 44-fold increase in fitting speed and a segmentation accuracy that is on par with Gauss-Newton optimization, one of the most widely used optimization algorithms for such problems. The second contribution involves an investigation on hierarchical 2D+time active shape models (ASMs), that integrate temporal constraints and simultaneously improve the 3D AAM based segmentation. We obtain encouraging results (endocardial/epicardial error 1.43+/-0.49 mm/1.51+/-0.48 mm) on 7980 short axis cardiac MR images acquired from 33 subjects. We have placed our dataset online, for the community to use and build upon.

Loach and Marí-Beffa (Vis Cogn, 10:513-526, 2003) observed that a distractor stimulus, presented immediately after a behaviorally relevant target stimulus, negatively primed a related probe stimulus indicating that the distractor had been inhibited. They argued that "post-target inhibition" may be a mechanism for preventing interference from temporally proximal stimuli; interference that could potentially result in a binding/intrusion error. In order to test this hypothesis, the authors carried out two rapid serial visual presentation (RSVP) experiments in which participants had to report either the identity (Experiment 1) or color (Experiment 2) of a target letter surrounded by distractor letters. In Experiment 1, a close relationship between priming and errors was observed. When a distractor stimulus showed evidence of being inhibited the participant was less likely to commit a binding error. The opposite was true when a distractor stimulus showed evidence of being facilitated. The results of Experiment 2 showed limited evidence of the same relationship.

Subjects were required to attend to an orientation and make judgments about the stripes on briefly presented disks. Stripe orientation was varied so that they could be at, near, or far from the attended orientation. According to the selective-tuning model (Tsotsos, 1990; Tsotsos et al., 1995), attending to an orientation results in an inhibitory surround for nearby orientations, but not for orientations farther away. In line with this prediction, the results revealed an inhibitory surround. As in the spatial domain, attending to a point in orientation space results in an inhibitory surround for nearby orientations.

Selective Tuning (ST) presents a framework for modeling attention and in this work we show how it performs in covert visual search tasks by comparing its performance to human performance. Two implementations of ST have been developed. The Object Recognition Model recognizes and attends to simple objects formed by the conjunction of various features and the Motion Model recognizes and attends to motion patterns. The validity of the Object Recognition Model was first tested by successfully duplicating the results of Nagy and Sanchez. A second experiment was aimed at an evaluation of the model's performance against the observed continuum of search slopes for feature-conjunction searches of varying difficulty. The Motion Model was tested against two experiments dealing with searches in the visual motion domain. A simple odd-man-out search for counter-clockwise rotating octagons among identical clockwise rotating octagons produced linear increase in search time with the increase of set size. The second experiment was similar to one described by Thorton and Gilden. The results from both implementations agreed with the psychophysical data from the simulated experiments. We conclude that ST provides a valid explanatory mechanism for human covert visual search performance, an explanation going far beyond the conventional saliency map based explanations.

The perception of changes in the direction of objects that translate in space is an important function of our visual system. Here we investigate the brain electrical phenomena underlying such a function by using a combination of magnetoencephalography (MEG) and magnetic resonance imaging. We recorded MEG-evoked responses in 9 healthy human subjects while they discriminated the direction of a transient change in a translationally moving random dot pattern presented either to the right or to the left of a central fixation point. We found that responses reached their maximum in 2 main regions corresponding to motion processing area middle temporal (MT)/V5 contralateral to the stimulated visual field, and to the right inferior parietal lobe (rIPL). The activation latencies were very similar in both regions ( approximately 135 ms) following the direction change onset. Our findings suggest that area MT/V5 provides the strongest sensory signal in response to changes in the direction of translational motion, whereas area rIPL may be involved either in the sensory processing of transient motion signals or in the processing of signals related to orienting of attention.

The spatial focus of attention has traditionally been envisioned as a simple spatial gradient of enhanced activity that falls off monotonically with increasing distance. Here, we show with high-density magnetoencephalographic recordings in human observers that the focus of attention is not a simple monotonic gradient but instead contains an excitatory peak surrounded by a narrow inhibitory region. To demonstrate this center-surround profile, we asked subjects to focus attention onto a color pop-out target and then presented probe stimuli at various distances from the target. We observed that the electromagnetic response to the probe was enhanced when the probe was presented at the location of the target, but the probe response was suppressed in a narrow zone surrounding the target and then recovered at more distant locations. Withdrawing attention from the pop-out target by engaging observers in a demanding foveal task eliminated this pattern, confirming a truly attention-driven effect. These results indicate that neural enhancement and suppression coexist in a spatially structured manner that is optimal to attenuate the most deleterious noise during visual object identification.

Cortical area MT/V5 in the human occipito-temporal cortex is activated by visual motion. In this study, we use functional imaging to demonstrate that a subregion of MT/V5 is more strongly activated by unidirectional motion with speed gradients than by other motion patterns. Our results suggest that like the monkey homolog middle temporal area (MT), human MT/V5 contains neurons selective for the processing of speed gradients. Such neurons may constitute an intermediate stage of processing between neurons selective for the average speed of unidirectional motion and neurons selective for different combinations of speed gradient and different motion directions such as expanding optical flow patterns.

The selective tuning model [Artif. Intell. 78 (1995) 507] is a neurobiologically plausible neural network model of visual attention. One of its key predictions is that to simultaneously solve the problems of convergence of neural input and selection of attended items, the portions of the visual neural network that process an attended stimulus must be surrounded by inhibition. To test this hypothesis, we mapped the attentional field around an attended location in a matching task where the subject's attention was directed to a cued target while the distance of a probe item to the target was varied systematically. The main result was that accuracy increased with inter-target separation. The observed pattern of variation of accuracy with distance provided strong evidence in favor of the critical prediction of the model that attention is actively inhibited in the immediate vicinity of an attended location.

Orthodontists have defined a certain number of characteristic points, or landmarks, on X-ray images of the human skull which are used to study growth or as a diagnostic aid. This work presents the first step toward an automatic extraction of these points. They are defined with respect to particular lines which are retrieved first. The original image is preprocessed with a prefiltering operator (median filter) followed by an edge detector (Mero-Vassy operator). A knowledge-based line-following algorithm is subsequently applied, involving a production system with organized sets of rules and a simple interpreter. The a priori knowledge implemented in the algorithm must take into account the fact that the lines represent biological shapes and can vary considerably from one patient to the next. The performance of the algorithm is judged with the help of objective quality criteria. Determination of the exact shapes of the lines allows the computation of the positions of the landmarks.