Search

Clark (in press) has offered a forceful defense of the “hierarchical prediction machine” (HPM) approach to the brain. Roughly, HPM suggests that brains are in the business of making sense of incoming information by generating top-down models aimed at providing the optimal fit for the input data. A better fit between the model and the data minimizes prediction error, which Clark – following Friston (e.g., Friston, 2010) – construes as tantamount to reducing surprisal, i.e., “the sub-personally computed implausibility of some sensory state given the model of the world” (p. 17). Notwithstanding the breadth of his defense, Clark’s case is entirely built upon research on perception, attention, and action, all of which are on-line cognitive processes. With practically no mention of offline cognition, the theoretical pretensions of the HPM approach, which Clark so vigorously defends as a “single unifying explanatory framework” (p. 61) in cognitive science, are questionable. I suggest that this conspicuous absence might be partially remedied, at least for the case of remembering, by looking at recent Bayesian accounts of memory retrieval developed after Anderson’s Adaptive Control of Thought-Rational (ACT-R) model (Anderson and Milson, 1989; Anderson, 1990; Anderson and Schooler, 1991, 2000). Specifically, I suggest that the ACT-R model can be read as describing how memory retrieval attempts to minimize prediction error when finding the optimal memory given the costs of its retrieval and the organism’s current needs. Originally, the ACT-R model stated that remembering is a cognitive operation whose costs are offset by the gains attained when retrieval is successful. As such, our adaptive memory system would search for a particular memory as long as the probability of recovering it given our current needs is greater than the costs of its retrieval. The ACT-R model captures this insight in Bayesian terms thus: let Hi be the hypothesis that a particular memory is needed during a particular context, and let E be the evidence for an element of said context. Then, P(Hi|E)αP(E|Hi)P(Hi) (1) where P(E|Hi) determines the likelihood ratio that E is the case given Hi (i.e., the context factor), and P(Hi) gives the prior probability that a particular memory will be needed (i.e., the history factor). For present purposes, two consequences that follow from this formulation are relevant. First, as Anderson and Milson (1989) remarked, given the multiplicity of elements present in a retrieval context, the likelihood ratio representing the context factor is best understood as the multiplicative product of all the likelihood ratios for every element of the context given Hi1. As a result, certain contextual elements are going to be better cues than others (i.e., representing a larger positive contribution to the overall product), as it is the case with elements that were present in the context of encoding (Craik and Tulving, 1975). The second thing to notice is that the prior probability, according to the ACT-R model, depends on the history of previous retrievals. Originally, Anderson and Milson (1989) noted that determining the history factor could be daunting, if not impossible, as one “would have to follow people about their daily lives, keeping a complete record of when they use various facts [and] such an objective study of human information is close to impossible” (p. 705). To get around this problem, Anderson and Schooler (1991) suggested extracting prior probabilities from the statistical distribution of existent databases that, according to them, would capture “coherent slices of the environment.” One such environmental database, for instance, contained 2 years worth of word usage in the New York Times headlines. They found that the odds that a particular word was used in a certain headline was inversely correlated to its having occurred in a previous headline, with the probability diminishing the more time had passed since its last usage. Importantly, Anderson and Schooler (1991) showed that this model could fit extant data on recency and frequency effects on memory retrieval remarkably well2. Taken together, the context and the history factors suggest that the probability that a certain memory will be needed in a particular context can be predicted from the probability that it has been needed in the recent past in relevantly similar contexts. From the point of view of Clark’s HPM approach then, context and history factors combine in a hierarchical model that tries to find the most predictable memory – i.e., that which minimizes prediction error – for a needed memory given a cue3. Notwithstanding Anderson and Schooler’s impressive results, priors based on statistical distributions of limited environments do not seem to capture the full complexity of human memory retrieval. Recently, however, Hemmer and Steyvers (2009b, see also Hemmer and Steyvers, 2009a) tried a different tack. They obtained the prior probability of remembering the size of a certain object from the statistical distribution of participant’s responses on a norming phase, in which relative size judgments on a number of objects had to be performed. Thus, instead of determining the prior by collecting data from the participant’s size-judgment behavior before the study, they did it via generating a probability distribution from the participants’ judgments themselves. This model allowed them to predict with remarkable accuracy hits and false alarms in a recognition test, as participant’s responses approximated the means of the prior distribution for each item. So far I’ve tried to draw parallels between ACT-R inspired Bayesian models on memory retrieval and Clark’s HPM approach as a way to show that his explanatory framework can be extended to an offline cognitive process such as remembering. But in so doing I intentionally drew a stark contrast between the way in which Anderson and Schooler obtained prior probabilities and likelihood rates, and the way in which Hemmer and Steyvers did, because I think this difference illustrates a difficult challenge for the HPM framework. According to Clark, although the HPM approach is primarily thought of as describing the way in which the brain aims at reducing surprisal at anon-agential level, these very same processes may help to understand the agent-level experience of surprise reduction – the experience of sensing a stimulus as the least surprising (“surprisal-ing”!, p. 47). But this agent/non-agent gap may be difficult to bridge. Indeed, Clark himself acknowledges this difficulty when he says: “[T]here seems to be a large disconnect between ‘surprisal’ (the implausibility of some sensory state given a model of the world) and agent-level surprise. This is evident from the simple fact that the percept that, overall, best minimizes surprisal (hence minimizes prediction errors) ‘for’ the brain may well be, for me the agent, some highly surprising and unexpected state of affairs” (p. 46). Nonetheless, Clark (in press) believes that the two levels “are easily reconciled” when one recognizes that what appears to the agent as a surprising event may just be, in reality, only improbable. The agent might not have been expecting to experience some mental content or another, but from the point of view of the brain, such a content may actually be perfectly predictable. I find Clark’s response unsatisfying, for this surprise-surprisal gap – this “surprising gap” – between the agent and the non-agent levels is likely to occur more often than Clark assumes, and the frequency of this occurrence puts pressure on Clark to come up with a clearer explanation as to how HPM can in fact illuminate cognition at the agent-level. Consider the two approaches to generating prior probabilities and likelihood ratios mentioned above. In the case of Anderson and Schooler, the approach is agent-independent, as it involves collecting probability distributions of frequency responses that are independent of the subject’s own frequency-judgments. Likewise, priors generated from data at the neural level, such as those referenced by Clark in his essay, are also agent-independent. Conversely, Hemmer and Steyver’s approach is paradigmatically agent-dependent, as it involves generating a probability distribution from the participant’s own frequency-judgments. However, we have plenty of evidence showing that what we think is most frequent does not always correspond to what it is actually most frequent (Tversky and Kahneman, 1973, but see Manis et al., 1993). Moreover, the agent/non-agent mismatch that gives rise to this “surprising gap” may actually occur even when there is no experience of surprise at the agent-level. It may occur, for instance, when there is a prediction mismatch due to independent processes of prior updating at the agential and non-agential levels. As a result, although models with agent-independent priors may be equally good at fitting data as models with agent-dependent priors, they need not be, and it is an open empirical question whether or not they do – a question that cannot be simply dismissed on a priori grounds, as Clark does. So it seems to me that studying this surprising gap is itself an exciting avenue for future research. Why are there percepts that may appear surprising to the agent? What are the conditions under which surprise reduction meets surprisal reduction? Are false alarms in perception or in recognition memory better predicted with agent-dependent or agent-independent priors? These, I think, are all interesting questions worthy of being examined, and for which the HPM needs to find an answer if it really attempts to be a “unifying explanatory framework” for both agent and non-agent level cognitive phenomena4.

Clark (in press) has argued that our brain uses hierarchical predictive coding and the use of this concept would enable us to understand the mental life of the organism. We agree that hierarchical predictive coding provides a good framework for developing further theories and models for different aspects of brain functioning including, perception, action, attention, and experience. An important but least understood aspect of human cognition is the “I” as an experiencer. A person perceives one's self as an agent or controller of actions through one's interactions with the environment. This influences the perception of the event itself. Events that are felt to be caused by self are perceived differently from other events. For example, subjective experience of flash lag effect reduces when the subject plays an active part in the event (Ichikawa and Masakura, 2010). Intentional binding between an action and a tone has been reported to become stronger when subjects experience greater control over the action (Moore et al., 2009). Clark has pointed out that models of predictive coding may shed some light on aspects of subjective experience of agency (possibly self) and enable us to computationally model these experiences. A prediction based model that deals with self is the Bayesian version of the comparator model (Fletcher and Frith, 2009). In this model, agency is based on the successful prediction of the organisms’ sensory consequences of action performed by the organism. This model has also been used as a major component in the predictive coding based model of conscious presence (Seth et al., 2012). While Clark in his paper has emphasized on the formation of subjective experiences, it is not immediately clear how self can be understood in the context of hierarchical predictive coding. One possible way to understand self and agency is through the notion of event-control (Jordan, 2003) which argues that self gets attached to a particular level of event-control at a given time. Using the example of driving, for an inexperienced driver, self would get attached to the proximal level of control related to hand or leg movements and the clutch, brake, and gear. However, an experienced driver for whom driving is fairly automatic self would get attached to the successful achievement of the goal, that is, driving from one place to another. Event-control approach proposes the notion of hierarchically arranged control loops (Jordan, 2003) that are possibly nested (Feinberg, 2011), with the distal goal at the top-level and the proximal sensory-motor loop at the lowest level. There is both a feed-forward and a feed-back information transfer between different levels in the hierarchy. Each level constrains the activity of the lower levels by fixing certain parameters (the lower level control loops function under the constraints provided by the higher control loop). In return what is fed back to the higher level is the amount of control that the lower level could achieve given the constraints. Using this information, the constraints are modified so as to achieve efficient control by the lower levels. In this approach, the subjective sense of self is not attached to a particular loop (Frith, 2005; Seth et al., 2012). It rather varies dynamically as a function of the successful control achieved at a particular level. The sense of self is conceptualized as fluid or plastic with self attached to the highest level in which control is successfully exercised at a particular level in the hierarchy. The level at which “self” gets attached could change as an organism continues to interact with the environment in the context of an intentional goal, moving across different levels in the hierarchy at different times. Empirical data from studies exploring the link between sense of self and event-control (Jordan, 2003; Kumar and Srinivasan, under review) suggest that participant's sense of self does vary with the amount of control that they can exercise. We have studied the emergence of self as a function of different levels in a event-control hierarchy using a multi-agent paradigm (Kumar and Srinivasan, under review). When control is manipulated at multiple levels, self gets attached to the higher/more distal level at which control can be successfully exercised. When the exercise of control is poor at a higher level, the sense of self attaches to the lower (more proximal) level of event-control. The results also show that when people misidentify as some other agent pursuing a common goal, misidentification is more when the other agent affords a greater experienced control (again in a hierarchical fashion). The results based on event-control approach is consistent with the studies that have shown a link between the feeling of control and the subjective experience of agency (Frith, 2005; Moore et al., 2009; Dewey et al., 2010; Desantis et al., 2011). The event-control approach goes beyond other hierarchical predictive coding models of subjective experience (Frith, 2005; Seth et al., 2012), not conceptualizing self in terms of what is happening at the pre-fixed top-level module in the hierarchy. It visualizes a dynamic fluid self that depends on control exercised at a particular level and changes as a function of an organism's interactions with the environment. This enables us to understand changes in self and identity as a function of interaction between organism and the environment (as more situated and embodied) rather than simply focusing on the organism per se. In addition, self is neither visualized as a cause of decisions made by the organism nor as epiphenomenal but rather in terms of experience that emerges out of regularities in perception-action events. To conclude, we broadly agree with the notion of hierarchical predictive coding and extend it to explain the emergence of self and the experience of agency by naturalizing them as the level of control achieved at a given level in the hierarchy involving perceptual-action events. The use of hierarchical control systems would enable us to study our mental life and better understand the epistemic concepts of intentionality, consciousness, and self.