Search

Chota and Van der Stigchel (this issue), Iamshchinina, Christophel, Gayet, and Rademaker (this issue), Lorenc and Sreenivasa (this issue), and Teng and Postle (this issue) each present a commentary regarding Xu (2020) where I conclude that sensory regions are nonessential for the storage of information in visual working memory (VWM). They argue instead that sensory regions are critical to VWM storage. Here I briefly reiterate some of the key evidence against this account, some of which has not been accounted by the four commentaries. I also provide a detailed reanalysis of why the main evidence supporting this account may be problematic. Collectively, existence evidence from human neuroimaging and TMS studies and that from monkey neurophysiology studies does not provide strong support for the sensory storage account of VWM. To form an accurate understanding of the distinctive role each brain region may play in perception and VWM as well as how they may interact to collectively support a VWM task, it is important that we properly survey and evaluate all the available evidence.

How do humans evaluate temporally accumulated discrete pieces of evidence and arrive at a decision despite the presence of conflicting evidence? In the present study, we showed human participants a sequential presentation of objects drawn from two novel object categories and asked them to decide whether a given presentation contained more objects from one or the other category. We found that both a more disparate ratio and greater numerosity of objects improved both reaction time (RT) and accuracy. The effect of numerosity was separate from ratio, where with a fixed object ratio, sequences with more total objects had lower RT and lower error rates than those with fewer total objects. We replicated these results across three experiments. Additionally, even with the total presentation duration equated and with the motor response assignment varied from trial to trial, an effect of numerosity was still found in RT. The same RT benefit was also present when objects were shown simultaneously, rather than sequentially. Together, these results showed that, for comparative numerosity judgment involving sequential displays, there was a benefit of numerosity, such that showing more objects independent of the object ratio and the total presentation time led to faster decision performance.

Recent work has highlighted the role of early visual areas in visual working memory (VWM) storage and put forward a sensory storage account of VWM. Using a distractor interference paradigm, however, we previolsy showed that the contribution of early visual areas to VWM storage may not be essential. Instead, higher cortical regions such as the posterior parietal cortex may play a more significant role in VWM storage. This is consistent with reviews of other available behavioral, neuroimaging and neurophysiology results. Recently, a number of studies brought forward new evidence regarding this debate. Here I review these new pieces of evidence in detail and show that there is still no strong and definitive evidence supporting an essential role of the early visual areas in VWM storage. Instead, converging evidence suggests that early visual areas may contribute to the decision stage of a VWM task by facilitating target and probe comparison. Aside from further clarifying this debate, it is also important to note that whether or not VWM storage uses a sensory code depends on how it is defined, and that behavioral interactions between VWM and perception tasks do not necessarily support the involvement of sensory regions in VWM storage.

Objects in the real world often appear with other objects. To recover the identity of an object whether or not other objects are encoded concurrently, in primate object-processing regions, neural responses to an object pair have been shown to be well approximated by the average responses to each constituent object shown alone, indicating the whole is equal to the average of its parts. This is present at the single unit level in the slope of response amplitudes of macaque IT neurons to paired and single objects, and at the population level in response patterns of fMRI voxels in human ventral object processing regions (e.g., LO). Here we show that averaging exists in both single fMRI voxels and voxel population responses in human LO, with better averaging in single voxels leading to better averaging in fMRI response patterns, demonstrating a close correspondence of averaging at the fMRI unit and population levels. To understand if a similar averaging mechanism exists in convolutional neural networks (CNNs) pretrained for object classification, we examined five CNNs with varying architecture, depth and the presence/absence of recurrent processing. We observed averaging at the CNN unit level but rarely at the population level, with CNN unit response distribution in most cases did not resemble human LO or macaque IT responses. The whole is thus not equal to the average of its parts in CNNs, potentially rendering the individual objects in a pair less accessible in CNNs during visual processing than they are in the human brain.

Forming transformation-tolerant object representations is critical to high-level primate vision. Despite its significance, many details of tolerance in the human brain remain unknown. Likewise, despite the ability of convolutional neural networks (CNNs) to exhibit human-like object categorization performance, whether CNNs form tolerance similar to that of the human brain is unknown. Here we provide the first comprehensive documentation and comparison of three tolerance measures in the human brain and CNNs. We measured fMRI responses from human ventral visual areas to real-world objects across both Euclidean and non-Euclidean feature changes. In single fMRI voxels in higher visual areas, we observed robust object response rank-order preservation across feature changes. This is indicative of functional smoothness in tolerance at the fMRI meso-scale level that has never been reported before. At the voxel population level, we found highly consistent object representational structure across feature changes towards the end of ventral processing. Rank-order preservation, consistency, and a third tolerance measure, cross-decoding success (i.e., a linear classifier’s ability to generalize performance across feature changes) showed an overall tight coupling. These tolerance measures were lower for Euclidean than non-Euclidean feature changes in lower visual areas, but increased over the course of ventral processing in most cases. These characteristics of tolerance, however, were absent in eight CNNs pretrained with ImageNet images with varying network architecture, depth, the presence/absence of recurrent processing, or whether a network was pretrained with the original or stylized ImageNet images that encouraged shape processing. Most notably, CNNs do not show increased representational consistency across feature changes at the higher layers. CNNs thus do not appear to develop the same kind of tolerance as the human brain over the course of visual processing.Significant StatementPerceiving object identity among changes in non-identity features and forming transformation-tolerant object representations is essential to high-level primate vision. Here we provide a comprehensive documentation and comparison of three tolerance measures between the human brain and CNNs pretrained for object classification. While all three measures show increased tolerance in the human brain across four types of feature changes towards the end of ventral visual processing, CNNs fail to develop the same kind of tolerance with visual processing.

A visual object is characterized by multiple visual features, including its identity, position and size. Despite the usefulness of identity and nonidentity features in vision and their joint coding throughout the primate ventral visual processing pathway, they have so far been studied relatively independently. Here in both female and male human participants, the coding of identity and nonidentity features was examined together across the human ventral visual pathway. The nonidentity features tested included two Euclidean features (position and size) and two non-Euclidean features (image statistics and spatial frequency (SF) content of an image). Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with identity outweighing the non-Euclidean but not the Euclidean features at higher levels of visual processing. In 14 convolutional neural networks (CNNs) pretrained for object categorization with varying architecture, depth, and with/without recurrent processing, nonidentity feature representation showed an initial large increase from early to mid-stage of processing, followed by a decrease at later stages of processing, different from brain responses. Additionally, from lower to higher levels of visual processing, position became more underrepresented and image statistics and SF became more overrepresented compared with identity in CNNs than in the human brain. Similar results were obtained in a CNN trained with stylized images that emphasized shape representations. Overall, by measuring the coding strength of object identity and nonidentity features together, our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs.SIGNIFICANCE STATEMENTThis study examined the coding strength of object identity and four types of nonidentity features along the human ventral visual processing pathway and compared brain responses with those of 14 convolutional neural networks (CNNs) pretrained to perform object categorization. Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with some notable differences among the different nonidentity features. CNNs differed from the brain in a number of aspects in their representations of identity and nonidentity features over the course of visual processing. Our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs.

Convolutional neural networks (CNNs) are increasingly used to model human vision due to their high object categorization capabilities and general correspondence with human brain responses. Here we evaluate the performance of 14 different CNNs compared with human fMRI responses to natural and artificial images using representational similarity analysis. Despite the presence of some CNN-brain correspondence and CNNs' impressive ability to fully capture lower level visual representation of real-world objects, we show that CNNs do not fully capture higher level visual representations of real-world objects, nor those of artificial objects, either at lower or higher levels of visual representations. The latter is particularly critical, as the processing of both real-world and artificial visual stimuli engages the same neural circuits. We report similar results regardless of differences in CNN architecture, training, or the presence of recurrent processing. This indicates some fundamental differences exist in how the brain and CNNs represent visual information.

Although the primate posterior parietal cortex (PPC) has been largely associated with space, attention, and action-related processing, a growing number of studies have reported the direct representation of a diverse array of action-independent nonspatial visual information in the PPC during both perception and visual working memory. By describing the distinctions and the close interactions of visual representation with space, attention, and action-related processing in the PPC, here I propose that we may understand these diverse PPC functions together through the unique contribution of the PPC to adaptive visual processing and form a more integrated and structured view of the role of the PPC in vision, cognition, and action.

Despite decades of neuroscience research, our understanding of the relationship between color and form processing in the primate ventral visual pathway remains incomplete. Using fMRI multivoxel pattern analysis, this study examined the coding of color with both a simple form feature (orientation) and a mid-level form feature (curvature) in human early visual areas V1 to V4, posterior and central color regions, and shape areas in ventral and lateral occipito-temporal cortex. With the exception of the central color region (which showed color but not form decoding), successful color and form decoding was found in all other regions examined, even for color and shape regions showing univariate sensitivity to one feature. That said, all regions exhibited significant feature decoding biases, with decoding from color and shape regions largely consistent with their univariate preferences. Color and form are thus represented in neither a completely distributed nor a completely modular manner, but a biased distributed manner. Interestingly, coding of one feature in a brain region was always tolerant to changes in the other feature, indicating relative independence of color and form coding throughout the ventral visual cortex. Although evidence for interactive coding of color and form also existed, the effect was weak and only existed for color and orientation conjunctions in early visual cortex. No evidence for interactive coding of color and curvature was found. The predominant relationship between color and form coding in the human brain appears to be one of anatomical coexistence (in a biased distributed manner), but representational independence.

In everyday life, we have no trouble categorizing objects varying in position, size, and orientation. Previous fMRI research shows that higher-level object processing regions in the human lateral occipital cortex may link object responses from different affine states (i.e. size and viewpoint) through a general linear mapping function capable of predicting responses to novel objects. In this study, we extended this approach to examine the mapping for both Euclidean (e.g. position and size) and non-Euclidean (e.g. image statistics and spatial frequency) transformations across the human ventral visual processing hierarchy, including areas V1, V2, V3, V4, ventral occipitotemporal cortex (VOT), and lateral occipitotemporal cortex (LOT). The predicted pattern generated from a linear mapping function could capture a significant amount of the changes associated with the transformations throughout the ventral visual stream. The derived linear mapping functions were not category independent, as performance was better for the categories included than those not included in training and better between two similar versus two dissimilar categories in both lower and higher visual regions. Consistent with object representations being stronger in higher than lower visual regions, pattern selectivity and object category representational structure were somewhat better preserved in the predicted patterns in higher than lower visual regions. There were no notable differences between Euclidean and non-Euclidean transformations. These findings demonstrate a near-orthogonal representation of object identity and these non-identity features throughout the human ventral visual processing pathway, with these non-identity features largely untangled from the identity features early in visual processing. Significance Statement Presently we still do not fully understand how object identity and non-identity (e.g. position, size) information are simultaneously represented in the primate ventral visual system to form invariant representations. Previous work suggests that the human lateral occipital cortex may be linking different affine states of object representations through general linear mapping functions. Here we show that across the entire human ventral processing pathway, we could link object responses in different states of non-identity transformations through linear mapping functions for both Euclidean and non-Euclidean transformations. These mapping functions are not identity-independent, suggesting that object identity and non-identity features are represented in a near, rather than a completely, orthogonal manner.

To interact with real-world objects, any effective visual system must jointly code the unique features defining each object. Despite decades of neuroscience research, we still lack a firm grasp on how the primate brain binds visual features. Here we apply a novel network-based stimulus-rich representational similarity approach to study color and shape binding in five convolutional neural networks (CNNs) with varying architecture, depth, and presence/absence of recurrent processing. All CNNs showed near-orthogonal color and shape processing in early layers, but increasingly interactive feature coding in higher layers, with this effect being much stronger for networks trained for object classification than untrained networks. These results characterize for the first time how multiple visual features are coded together in CNNs. The approach developed here can be easily implemented to characterize whether a similar coding scheme may serve as a viable solution to the binding problem in the primate brain.

Any given visual object input is characterized by multiple visual features, such as identity, position and size. Despite the usefulness of identity and nonidentity features in vision and their joint coding throughout the primate ventral visual processing pathway, they have so far been studied relatively independently. Here we document the relative coding strength of object identity and nonidentity features in a brain region and how this may change across the human ventral visual pathway. We examined a total of four nonidentity features, including two Euclidean features (position and size) and two non-Euclidean features (image statistics and spatial frequency content of an image). Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with identity outweighed the non-Euclidean features, but not the Euclidean ones, in higher levels of visual processing. A similar analysis was performed in 14 convolutional neural networks (CNNs) pretrained to perform object categorization with varying architecture, depth, and with/without recurrent processing. While the relative coding strength of object identity and nonidentity features in lower CNN layers matched well with that in early human visual areas, the match between higher CNN layers and higher human visual regions were limited. Similar results were obtained regardless of whether a CNN was trained with real-world or stylized object images that emphasized shape representation. Together, by measuring the relative coding strength of object identity and nonidentity features, our approach provided a new tool to characterize feature coding in the human brain and the correspondence between the brain and CNNs.SIGNIFICANCE STATEMENTThis study documented the relative coding strength of object identity compared to four types of nonidentity features along the human ventral visual processing pathway and compared brain responses with those of 14 CNNs pretrained to perform object categorization. Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with the coding strength of the different nonidentity features differed at higher levels of visual processing. While feature coding in lower CNN layers matched well with that of early human visual areas, the match between higher CNN layers and higher human visual regions were limited. Our approach provided a new tool to characterize feature coding in the human brain and the correspondence between the brain and CNNs.

Convolutional neural networks (CNNs) have achieved very high object categorization performance recently. It has increasingly become a common practice in human fMRI research to regard CNNs as working model of the human visual system. Here we reevaluate this approach by comparing fMRI responses from the human brain in three experiments with those from 14 different CNNs. Our visual stimuli included original and filtered versions of real-world object images and images of artificial objects. Replicating previous findings, we found a brain-CNN correspondence in a number of CNNs with lower and higher levels of visual representations in the human brain better resembling those of lower and higher CNN layers, respectively. Moreover, the lower layers of some CNNs could fully capture the representational structure of human early visual areas for both the original and filtered real-world object images. Despite these successes, no CNN examined could fully capture the representational structure of higher human visual processing areas. They also failed to capture that of artificial object images in all levels of visual processing. The latter is particularly troublesome, as decades of vision research has demonstrated that the same algorithms used in the processing of natural images would support the processing of artificial visual stimuli in the primate brain. Similar results were obtained when a CNN was trained with stylized object images that emphasized shape representation. CNNs likely represent visual information in fundamentally different ways from the human brain. Current CNNs thus may not serve as sound working models of the human visual system.Significance StatementRecent CNNs have achieved very high object categorization performance, with some even exceeding human performance. It has become common practice in recent neuroscience research to regard CNNs as working models of the human visual system. Here we evaluate this approach by comparing fMRI responses from the human brain with those from 14 different CNNs. Despite CNNs’ ability to successfully perform visual object categorization like the human visual system, they appear to represent visual information in fundamentally different ways from the human brain. Current CNNs thus may not serve as sound working models of the human visual system. Given the current dominating trend of incorporating CNN modeling in visual neuroscience research, our results question the validity of such an approach.

Recent human fMRI pattern-decoding studies have highlighted the involvement of sensory areas in visual working memory (VWM) tasks and argue for a sensory account of VWM storage. In this review, evidence is examined from human behavior, fMRI decoding, and transcranial magnetic stimulation (TMS) studies, as well as from monkey neurophysiology studies. Contrary to the prevalent view, the available evidence provides little support for the sensory account of VWM storage. Instead, when the ability to resist distraction and the existence of top-down feedback are taken into account, VWM-related activities in sensory areas seem to reflect feedback signals indicative of VWM storage elsewhere in the brain. Collectively, the evidence shows that prefrontal and parietal regions, rather than sensory areas, play more significant roles in VWM storage.

Recent studies have challenged the ventral/"what" and dorsal/"where" two-visual-processing-pathway view by showing the existence of "what" and "where" information in both pathways. Is the two-pathway distinction still valid? Here, we examined how goal-directed visual information processing may differentially impact visual representations in these two pathways. Using fMRI and multivariate pattern analysis, in three experiments on human participants (57% females), by manipulating whether color or shape was task-relevant and how they were conjoined, we examined shape-based object category decoding in occipitotemporal and parietal regions. We found that object category representations in all the regions examined were influenced by whether or not object shape was task-relevant. This task effect, however, tended to decrease as task-relevant and irrelevant features were more integrated, reflecting the well-known object-based feature encoding. Interestingly, task relevance played a relatively minor role in driving the representational structures of early visual and ventral object regions. They were driven predominantly by variations in object shapes. In contrast, the effect of task was much greater in dorsal than ventral regions, with object category and task relevance both contributing significantly to the representational structures of the dorsal regions. These results showed that, whereas visual representations in the ventral pathway are more invariant and reflect "what an object is," those in the dorsal pathway are more adaptive and reflect "what we do with it." Thus, despite the existence of "what" and "where" information in both visual processing pathways, the two pathways may still differ fundamentally in their roles in visual information representation.SIGNIFICANCE STATEMENT Visual information is thought to be processed in two distinctive pathways: the ventral pathway that processes "what" an object is and the dorsal pathway that processes "where" it is located. This view has been challenged by recent studies revealing the existence of "what" and "where" information in both pathways. Here, we found that goal-directed visual information processing differentially modulates shape-based object category representations in the two pathways. Whereas ventral representations are more invariant to the demand of the task, reflecting what an object is, dorsal representations are more adaptive, reflecting what we do with the object. Thus, despite the existence of "what" and "where" information in both pathways, visual representations may still differ fundamentally in the two pathways.

Based on different cognitive tasks and mapping methods, the human intraparietal sulcus (IPS) has been subdivided according to multiple different organizational schemes. The presence of topographically organized regions throughout IPS indicates a strong location-based processing in this brain region. However, visual short-term memory (VSTM) studies have shown that while a region in the inferior IPS region (inferior IPS) is involved in object individuation and selection based on location, a region in the superior IPS (superior IPS) primarily encodes and stores object featural information. Here, we determined the localization of these two VSTM IPS regions with respect to the topographic IPS regions in individual participants and the role of different IPS regions in location- and feature-based processing. Anatomically, inferior IPS showed an 85.2% overlap with topographic IPS regions, with the greatest overlap seen in V3A and V3B, and superior IPS showed a 73.6% overall overlap, with the greatest overlap seen in IPS0-2. Functionally, there appeared to be a partial overlap between IPS regions involved in location- and feature-based processing, with more inferior and medial regions showing a stronger location-based processing and more superior and lateral regions showing a stronger feature-based processing. Together, these results suggest that understanding the multiplex nature of IPS in visual cognition may not be reduced to examining the functions of the different IPS topographic regions, but rather, it can only be accomplished by understanding how regions identified by different tasks and methods may colocalize with each other.

The representation of object identity is fundamental to human vision. Using fMRI and multivoxel pattern analysis, here we report the representation of highly abstract object identity information in human parietal cortex. Specifically, in superior intraparietal sulcus (IPS), a region previously shown to track visual short-term memory capacity, we found object identity representations for famous faces varying freely in viewpoint, hairstyle, facial expression, and age; and for well known cars embedded in different scenes, and shown from different viewpoints and sizes. Critically, these parietal identity representations were behaviorally relevant as they closely tracked the perceived face-identity similarity obtained in a behavioral task. Meanwhile, the task-activated regions in prefrontal and parietal cortices (excluding superior IPS) did not exhibit such abstract object identity representations. Unlike previous studies, we also failed to observe identity representations in posterior ventral and lateral visual object-processing regions, likely due to the greater amount of identity abstraction demanded by our stimulus manipulation here. Our MRI slice coverage precluded us from examining identity representation in anterior temporal lobe, a likely region for the computing of identity information in the ventral region. Overall, we show that human parietal cortex, part of the dorsal visual processing pathway, is capable of holding abstract and complex visual representations that are behaviorally relevant. These results argue against a “content-poor” view of the role of parietal cortex in attention. Instead, the human parietal cortex seems to be “content rich” and capable of directly participating in goal-driven visual information representation in the brain.SIGNIFICANCE STATEMENTThe representation of object identity (including faces) is fundamental to human vision and shapes how we interact with the world. Although object representation has traditionally been associated with human occipital and temporal cortices, here we show, by measuring fMRI response patterns, that a region in the human parietal cortex can robustly represent task-relevant object identities. These representations are invariant to changes in a host of visual features, such as viewpoint, and reflect an abstract level of representation that has not previously been reported in the human parietal cortex. Critically, these neural representations are behaviorally relevant as they closely track the perceived object identities. Human parietal cortex thus participates in the moment-to-moment goal-directed visual information representation in the brain.

Recent studies have provided conflicting accounts regarding where in the human brain visual short-term memory (VSTM) content is stored, with strong univariate fMRI responses reported in superior intraparietal sulcus (IPS) but robust multivariate decoding reported in occipital cortex. Given the continuous influx of information in everyday vision, VSTM storage under distraction is often required. We found that neither distractor presence nor predictability during the memory delay affected behavioral performance. Similarly, superior IPS exhibited consistent decoding of VSTM content across all distractor manipulations and had multivariate responses that closely tracked behavioral VSTM performance. However, occipital decoding of VSTM content was significantly modulated by distractor presence and predictability. Furthermore, we found no effect of target-distractor similarity on VSTM behavioral performance, further challenging the role of sensory regions in VSTM storage. Overall, consistent with previous univariate findings, these results show that superior IPS, not occipital cortex, plays a central role in VSTM storage.

Primate ventral and dorsal visual pathways both contain visual object representations. Dorsal regions receive more input from magnocellular system while ventral regions receive inputs from both magnocellular and parvocellular systems. Due to potential differences in the spatial sensitivites of manocellular and parvocellular systems, object representations in ventral and dorsal regions may differ in how they represent visual input from different spatial scales. To test this prediction, we asked observers to view blocks of images from six object categories, shown in full spectrum, high spatial frequency (SF), or low SF. We found robust object category decoding in all SF conditions as well as SF decoding in nearly all the early visual, ventral, and dorsal regions examined. Cross-SF decoding further revealed that object category representations in all regions exhibited substantial tolerance across the SF components. No difference between ventral and dorsal regions was found in their preference for the different SF components. Further comparisons revealed that, whereas differences in the SF component separated object category representations in early visual areas, such a separation was much smaller in downstream ventral and dorsal regions. In those regions, variations among the object categories played a more significant role in shaping the visual representational structures. Our findings show that ventral and dorsal regions are similar in how they represent visual input from different spatial scales and argue against a dissociation of these regions based on differential sensitivity to different SFs.

Visual information processing contains two opposite needs. There is both a need to comprehend the richness of the visual world and a need to extract only pertinent visual information to guide thoughts and behavior at a given moment. I argue that these two aspects of visual processing are mediated by two complementary visual systems in the primate brain—specifically, the occipitotemporal cortex (OTC) and the posterior parietal cortex (PPC). The role of OTC in visual processing has been documented extensively by decades of neuroscience research. I review here recent evidence from human imaging and monkey neurophysiology studies to highlight the role of PPC in adaptive visual processing. I first document the diverse array of visual representations found in PPC. I then describe the adaptive nature of visual representation in PPC by contrasting visual processing in OTC and PPC and by showing that visual representations in PPC largely originate from OTC.

Recent studies have demonstrated the existence of rich visual representations in both occipitotemporal cortex (OTC) and posterior parietal cortex (PPC). Using fMRI decoding and a bottom-up data-driven approach, we showed that although robust object category representations exist in both OTC and PPC, there is an information-driven 2-pathway separation among these regions in the representational space, with occipitotemporal regions arranging hierarchically along 1 pathway and posterior parietal regions along another pathway. We obtained 10 independent replications of this 2-pathway distinction, accounting for 58–81% of the total variance of the region-wise differences in visual representation. The separation of the PPC regions from higher occipitotemporal regions was not driven by a difference in tolerance to changes in low-level visual features, did not rely on the presence of special object categories, and was present whether or not object category was task relevant. Our information-driven 2-pathway structure differs from the well-known ventral-what and dorsal-where/how characterization of posterior brain regions. Here both pathways contain rich nonspatial visual representations. The separation we see likely reflects a difference in neural coding scheme used by PPC to represent visual information compared with that of OTC.

Visual object expertise correlates with neural selectivity in the fusiform face area (FFA). Although behavioral studies suggest that visual expertise is associated with increased use of holistic and configural information, little is known about the nature of the supporting neural representations. Using high-resolution 7-T functional magnetic resonance imaging, we recorded the multivoxel activation patterns elicited by whole cars, configurally disrupted cars, and car parts in individuals with a wide range of car expertise. A probabilistic support vector machine classifier was trained to differentiate activation patterns elicited by whole car images from activation patterns elicited by misconfigured car images. The classifier was then used to classify new combined activation patterns that were created by averaging activation patterns elicited by individually presented top and bottom car parts. In line with the idea that the configuration of parts is critical to expert visual perception, car expertise was negatively associated with the probability of a combined activation pattern being classified as a whole car in the right anterior FFA, a region critical to vision for categories of expertise. Thus, just as found for faces in normal observers, the neural representation of cars in right anterior FFA is more holistic for car experts than car novices, consistent with common mechanisms of neural selectivity for faces and other objects of expertise in this area.

Most of human daily social interactions rely on the ability to successfully recognize faces. Yet ∼2% of the human population suffers from face blindness without any acquired brain damage [this is also known as developmental prosopagnosia (DP) or congenital prosopagnosia]). Despite the presence of severe behavioral face recognition deficits, surprisingly, a majority of DP individuals exhibit normal face selectivity in the right fusiform face area (FFA), a key brain region involved in face configural processing. This finding, together with evidence showing impairments downstream from the right FFA in DP individuals, has led some to argue that perhaps the right FFA is largely intact in DP individuals. Using fMRI multivoxel pattern analysis, here we report the discovery of a neural impairment in the right FFA of DP individuals that may play a critical role in mediating their face-processing deficits. In seven individuals with DP, we discovered that, despite the right FFA's preference for faces and it showing decoding for the different face parts, it exhibited impaired face configural decoding and did not contain distinct neural response patterns for the intact and the scrambled face configurations. This abnormality was not present throughout the ventral visual cortex, as normal neural decoding was found in an adjacent object-processing region. To our knowledge, this is the first direct neural evidence showing impaired face configural processing in the right FFA in individuals with DP. The discovery of this neural impairment provides a new clue to our understanding of the neural basis of DP.