Search

Running behavioral studies online is a challenge as well as an opportunity we have been facing for several years. It allows collecting data quickly and from a much more diverse population compared to physical data collection. The Covid-19 pandemic pushed forward the development of guidelines based on experiences of success and failures. There are many considerations to take for getting the most benefit from collecting data online. We share hereby our own experience referring to research questions, software environment, participants, and data quality control, in the sincere hope to help others who are interested in utilizing online data collection for behavioral research.

Hindsight bias (HB) is the tendency to see known information as obvious. We studied metacognitive hindsight bias (MC-HB)-a shift away from one's original confidence regarding answers provided before learning the actual facts. In two experiments, participants answered general-knowledge questions in social scenarios and provided their confidence in each answer. Subsequently, they learned answers to half the questions and then recalled their initial answers and confidence. Finally, they reanswered, as a learning check. We measured confidence accuracy by calibration (over/underconfidence) and resolution (discrimination between incorrect and correct answers), expecting them to improve in hindsight. In both experiments, participants displayed robust HB and MC-HB for resolution despite attempts to recall the initial confidence in one's answer. In Experiment 2, promising anonymity to participants eliminated MC-HB, while social scenarios produced MC-HB for both resolution and calibration-indicative of overconfidence. Overall, our findings highlight that in social contexts, recall of confidence in hindsight is more consistent with answers' accuracy than confidence initially was. Social scenarios differently affect HB and MC-HB, thus dissociating these two biases.

Confidence and its accuracy have been most commonly examined in domains such as general knowledge and learning, with less study of other domains, such as applied knowledge and problem solving. Monitoring accuracy in real-world competencies may depend on characteristics of the domain. In this study, we examined whether monitoring accuracy, both calibration (resistance to overconfidence) and resolution (discrimination) indices, are stable within individuals across tasks that represent highly diverse domains. We examined the well-established domain of general knowledge and three understudied applied domains of financial calculation, probability calculation, and the social skill of emotion recognition. In addition, we examined correlations between monitoring accuracy and cognitive abilities (intellectual ability and working memory) and several aggregated judgments regarding each task as a whole (ratings of predicted performance, task difficulty, and effort required) as well as the classic postdictive itemized confidence ratings. We found that resistance to overconfidence (calibration) was significantly positively correlated across tasks, reflecting a confidence trait, but not resolution. We also found that cognitive abilities were more consistently predictive of calibration than of resolution. Aggregated judgments and postdictive confidence were significant predictors of both calibration and resolution, but associations were task specific. Emotion recognition displayed the most unique profile of findings relative to other tasks. We conclude that when considering a wide range of domains, calibration displays domain generality, but resolution may display specificity across tasks.

Meta-reasoning refers to processes by which people monitor problem-solving activities and regulate effort investment. Solving is hypothesized to begin with an initial Judgment of Solvability (iJOS)—the solver’s first impression as to whether the problem is solvable—which guides solving attempts. Meta-reasoning research has largely neglected non-verbal problems. In this study we used Raven’s matrices to examine iJOS in non-verbal problems and its predictive value for effort investment, final Judgment of Solvability (fJOS), and confidence in the final answer. We generated unsolvable matrix versions by switching locations of elements in original Raven’s matrices, thereby breaking the rules while keeping the original components. Participants provided quick (4 s) iJOSs for all matrices and then attempted to solve them without a time limit. In two experiments, iJOS predicted solving time, fJOS, and confidence. Moreover, although difficulty of the original matrices was dissociated from solvability, iJOS was misled by original matrix difficulty. Interestingly, when the unsolvable matrices were relatively similar to the originals (Experiment 2), iJOSs were reliable, discriminating between solvable and unsolvable matrices. When the unsolvable matrices involved greater disruption of the rules (Experiment 1), iJOS was not consistently predictive of solvability. This study addresses a gap in meta-reasoning research by highlighting the predictive value of iJOS for the solving processes that follow. The study also provides many future directions for meta-reasoning research in general, and regarding non-verbal problems, in particular.

We present InCognitoMatch, the first cognitive-aware crowdsourcing application for matching tasks. InCognitoMatch provides a handy tool to validate, annotate, and correct correspondences using the crowd whilst accounting for human matching biases. In addition, InCognitoMatch enables system administrators to control context information visible for workers and analyze their performance accordingly. For crowd workers, InCognitoMatch is an easy-to-use application that may be accessed from multiple crowdsourcing platforms. In addition, workers completing a task are offered suggestions for followup sessions according to their performance in the current session. For this demo, the audience will be able to experience InCognitoMatch thorough three use-cases, interacting with system as workers and as administrators.

Paper-and-pencil learning and testing are gradually shifting to computerized environments. Cognitive and metacognitive researchers find screen inferiority compared to paper in effort regulation, test performance, and extent of overconfidence, in some cases, with unknown differentiating factors. Notably, these studies used reading comprehension tasks involving lengthy texts, which confound technology-related and cognitive factors. We hypothesized that the medium provides a contextual cue which leads to shallower processing on screen regardless of text length, particularly when task characteristics hint that shallow processing is legitimate. To test this hypothesis, we used briefly phrased yet challenging problems for solving on screen or on paper. In Experiment 1, the time frame for solving the problems was manipulated. As with lengthy texts, only time pressure resulted in screen inferiority. In Experiment 2, under a loose time frame, the same problems were now framed as a preliminary task performed before a main problem-solving task. Only the initial task, with reduced perceived importance, revealed screen inferiority similarly to time pressure. In Experiment 3, we replicated Experiment 1's time frame manipulation, using a problem-solving task which involved reading only three isolated words. Screen inferiority in overconfidence was found again only under time pressure. The results suggest that metacognitive processes are sensitive to contextual cues that hint at the expected depth of processing, regardless of the reading burden involved.

The relationship between affect and metacognitive processes has been largely overlooked in both the affect and the metacognition literatures. While at the core of many affect-cognition theories is the notion that positive affective states lead people to be more confident, few studies systematically investigated how positive affect influences confidence and strategic behaviour. In two experiments, when participants were free to control answer interval to general knowledge questions (e.g."in what year", answer: "it was between 1970 and 1985"), participants induced with positive affect outperformed participants in a neutral affect condition. However, in Experiment 1 positive affect participants showed larger overconfidence than neutral affect participants. In Experiment 2, enhanced salience of social cues eliminated this overconfidence disadvantage of positive affect relative to neutral affect participants, without compromising their enhanced performance. Notably, in both experiments, positive affect led to compromised social norms regarding the answers' informativeness. Implications for both affect and metacognition are discussed.

Metacognitive research is dominated by meta-memory studies; meta-reasoning research is nascent. Accessibility – the number of associations for a stimulus – is a reliable heuristic cue for Feeling o...

Aim/Purpose: These days educators are expected to integrate technological tools into classes. Although they acquire relevant skills, they are often reluctant to use these tools. Background: We incorporated online forums for generating a Community of Inquiry (CoI) in a faculty development program. Extending the Technology, Pedagogy, and Content Knowledge (TPACK) model with Assessment Knowledge and content analysis of forum discourse and reflection after each CoI, we offer the Diagnostic Tool for Learning, Assessment, and Research (DTLAR). Methodology: This study spanned over two cycles of a development program for medical faculty. Contribution: This study demonstrates how the DTLAR supports in-depth examination of the benefits and challenges of using CoIs for learning and teaching. Findings: Before the program, participants had little experience with, and were reluctant to use, CoIs in classes. At the program completion, many were willing to adopt CoIs and appreciated this method’s contribution. Both CoIs discourse and reflections included positive attitudes regarding cognitive and teacher awareness categories. However, negative attitudes regarding affective aspects and time-consuming aspects of CoIs were exposed. Participants who experienced facilitating a CoI gained additional insights into its usefulness. Recommendations for Practitioners : The DTLAR allows analyzing adaption of online forums for learning and teaching. Recommendation for Researchers: The DTLAR allows analyzing factors that affect the acceptance of online fo-rums for learning and teaching. Impact on Society : While the tool was implemented in the context of medical education, it can be readily applied in other adult learning programs. Future Research: The study includes several design aspects that probably affected the improve-ment and challenges we found. Future research is called for providing guidelines for identifying boundary conditions and potential for further improvement.

Going beyond the origins of cognitive biases, which have been the focus of continued research, the notion of metacognitive myopia refers to the failure to monitor, control, and correct for biased inferences at the metacognitive level. Judgments often follow the given information uncritically, even when it is easy to find out or explicitly explained that information samples are misleading or invalid. The present research is concerned with metacognitive myopia in judgments of change. Participants had to decide whether pairs of binomial samples were drawn from populations with decreasing, equal, or increasing proportions p of a critical feature. Judgments of p changes were strongly affected by changes in absolute sample size n, such that only increases (decreases) in p that came along with increasing (decreasing) n were readily detected. Across 4 experiments these anomalies persisted even though the distinction of p and n was strongly emphasized through outcome feedback and full debriefing (Experiment 1-4), simultaneous presentation (Experiments 2-4), and recoding of experienced samples into descriptive percentages (Experiment 3-4). In Experiment 4, a joint attempt was made by 10 scientists working in 7 different institutions to develop an effective debiasing training, suggesting how multilab-collaboration might improve the quality of science in the early stage of operational research designing. Despite significant improvements in change judgments, debiasing treatments did not eliminate the anomalies. Possible ways of dealing with the metacognitive deficit are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Understanding processes that lead people to invest a certain amount of time in challenging tasks is important for theory and practice. In particular, researchers often assume strong linear associations between confidence, consensuality (the degree to which an answer is independently given by multiple participants), and response time. The Diminishing Criterion Model (DCM; Ackerman, 2014) is a metacognitive model which explains the stopping rules people employ under uncertainty in terms of the confidence-time association. This model is unique in predicting a curvilinear rather than a linear confidence-time association. Using consensuality as an alternative to confidence for predicting response time offers theoretical and practical opportunities. In four experiments, including replications and variations, we examined confidence (where collected) and consensuality as predictors of the time people invest in three problem-solving tasks and in real-life web searching. The results using consensuality, like those for confidence, fitted the curvilinear time pattern predicted by the DCM, with one exception: at least 30% of the population must endorse a potential answer for consensuality to predict response time based on the stopping rules in the DCM. Beyond examining consensuality as a predictor, the study brings converging evidence supporting the DCM's curvilinear confidence-time association over alternative models. The methodology used for analyzing web searching offers new directions for metacognitive research in naturally-performed tasks.

The schema matching problem is at the basis of integrating structured and semi-structured data. Being investigated in the fields of databases, AI, semantic Web and data mining for many years, the core challenge still remains the ability to create quality matchers, automatic tools for identifying correspondences among data concepts (e.g., database attributes). In this work, we investigate human matchers behavior using a new concept termed match consistency and introduce a novel use of cognitive models to explain human matcher performance. Using empirical evidence, we further show that human matching suffers from predictable biases when matching schemata, which prevent them from providing consistent matching.

There is an agreement that perceived usability is important beyond actual effectiveness of software systems. Perceived usability is often obtained by self-reports provided after system use. Aiming to improve summative usability testing, we propose a methodology to enhance in-depth testing of users' performance and perceived usability at the task level. The metacognitive research approach allows detailed analysis of cognitive processes. Adapting its methodologies, we propose the Metacognitive Usability Profile (MUP) which includes a comprehensive set of measures based on collecting confidence in the success of each particular task and triangulating it with objective measures. We demonstrate using the MUP by comparing two versions of a project management system. Based on a task analysis we allocated tasks that differ between the versions and let participants (N?=?100) use both versions. Although no difference was found between the versions in system-level perceived usability, the detailed task-level analysis exposed many differences. In particular, overconfidence was associated with low performance, which suggests that user interfaces better avoid illusions of knowing. Overall, the study demonstrates how the MUP exposes challenges users face. This, in turn, allows choosing the better task implementation among the examined options and to focus attempts for usability improvement. Metacognitive approach allows a detailed analysis of perceived usability and performance.We propose the Metacognitive Usability Profile (MUP) for a detailed usability test.Test output is a scorecard that allows task-level comparisons of user interfaces.We demonstrate MUP use by testing two versions of a project management system.Conclusion: MUP exposed differences otherwise concealed.

How accurate are individuals in judging the originality of their own ideas? Most metacognitive research has focused on well-defined tasks, such as learning, memory, and problem solving, providing limited insight into ill-defined tasks. The present study introduces a novel metacognitive self-judgment of originality, defined as assessments of the uniqueness of an idea in a given context. In three experiments, we examined the reliability, potential biases, and factors affecting originality judgments. Using an ideation task, designed to assess the ability to generate multiple divergent ideas, we show that people accurately acknowledge the serial order effect-judging later ideas as more original than earlier ideas. However, they systematically underestimate their ideas' originality. We employed a manipulation for affecting actual originality level, which did not affect originality judgments, and another one designed to affect originality judgments, which did not affect actual originality performance. This double dissociation between judgments and performance calls for future research to expose additional factors underlying originality judgments.

Screen inferiority in performance and metacognitive processes has been repeatedly found with text learning. Common explanations for screen inferiority relate to technological and physiological disadvantages associated with extensive reading on screen. However, recent studies point to lesser recruitment of mental effort on screen than on paper. Learning tasks involving a heavy reading burden confound technological and physiological media differences with potential media effects on recruitment of mental effort. The present study focused on media effects on effort recruitment. We examined whether screen inferiority remains even with a brief task that nevertheless requires effort recruitment. In two experiments, participants faced three short math problems that require systematic processing to solve correctly. We examined media effect on solving these problems, and the potential of disturbed perceptual fluency (i.e., disfluent versus fluent fonts) to induce effort investment. Overall, there were no performance differences between the media. However, when collecting confidence ratings, disfluency improved performance on screen and hindered it on paper. Only on paper confidence ratings were sensitive to performance differences associated with fluency, and resolution was better with the disfluent font than with the fluent font. Correspondingly, another sample reported on their preference of media for solving the problems. They expressed a clear reluctance to working on screen despite the task being brief. This preference is suggestive of reliable meta-metacognitive judgments reflecting the general lower quality of metacognitive processes on screen. The findings call for considering medium and presentation format effects on metacognitive processing when designing computerized environments, even for brief tasks.

Prior research suggests that reducing font clarity can cause people to consider printed information more carefully. The most famous demonstration showed that participants were more likely to solve counterintuitive math problems when they were printed in hard-to-read font. However, after pooling data from that experiment with 16 attempts to replicate it, we find no effect on solution rates. We examine potential moderating variables, including cognitive ability, presentation format, and experimental setting, but we find no evidence of a disfluent font benefit under any conditions. More generally, though disfluent fonts slightly increase response times, we find little evidence that they activate analytic reasoning.

Learners often allocate more study time to challenging items than to easier ones. Nevertheless, both predicted and actual memory performance are typically worse for difficult than for easier items. The resulting inverse relations between people’s predictions of their memory performance (judgments of learning; JOLs) and self-paced study time (ST) are often explained by bottom-up, data-driven ST allocation that is based on fluency. However, we demonstrate robust inverted U-shaped relations between JOLs and ST that cannot be explained by data-driven ST allocation alone. Consequently, we explored how two models of top-down, strategic ST allocation account for curvilinear JOL-ST relations. First, according to the Region of Proximal Learning model, people stop quickly on items for which they experience too little progress in learning. Second, according to the Diminishing Criterion Model, people set a time limit and stop studying when this time limit is reached. In three experiments, we manipulated motivation with different methods and examined which model best described JOL-ST relations. Consistent with the Diminishing Criterion Model but not with the Region of Proximal Learning model, results revealed that curvilinearity was due to people setting a time limit.

Meta-Reasoning refers to the processes that monitor the progress of our reasoning and problem-solving activities and regulate the time and effort devoted to them. Monitoring processes are usually experienced as feelings of certainty or uncertainty about how well a process has, or will, unfold. These feelings are based on heuristic cues, which are not necessarily reliable. Nevertheless, we rely on these feelings of (un)certainty to regulate our mental effort. Most metacognitive research has focused on memorization and knowledge retrieval, with little attention paid to more complex processes, such as reasoning and problem solving. In that context, we recently developed a Meta-Reasoning framework, used here to review existing findings, consider their consequences, and frame questions for future research.

Metacognitive monitoring that accompanies a learning task reflects self-prediction of achievement at test. Well-calibrated monitoring is important because it is by this subjective assessment that people allocate their learning efforts. Previous studies that compared learning outcomes and calibration of monitoring when learning texts on screen and on paper have found screen inferiority: screen learners performed worse and were more overconfident about their success. However, learning from one’s preferred medium was associated with attenuated overconfidence. The present study examined two methods for overcoming screen inferiority in these respects. First, practicing the study-test task allowed overcoming screen inferiority, but only among those who preferred reading from screens. Second, in-depth processing was encouraged by having participants generate keywords at a delay, before monitoring their knowledge and taking the test. This method eliminated screen inferiority even for the first-studied texts, but after practicing it, screen inferiority was re-exposed among those who preferred studying on paper. This study makes a practical contribution to educational practice by suggesting directions for overcoming screen inferiority. From a broader perspective, the study demonstrates that experience with the task and in-depth processing can attenuate overconfidence and that the effectiveness of learning-enhancing methods depends on the study context and learners’ preferences.

According to the Discrepancy Reduction Model for metacognitive regulation, people invest time in cognitive tasks in a goal-driven manner until their metacognitive judgment, either judgment of learning (JOL) or confidence, meets their preset goal. This stopping rule should lead to judgments above the goal, regardless of invested time. However, in many tasks, time is negatively correlated with JOL and confidence, with low judgments after effortful processing. This pattern has often been explained as stemming from bottom-up fluency effects on the judgments. While accepting this explanation for simple tasks, like memorizing pairs of familiar words, the proposed Diminishing Criterion Model (DCM) challenges this explanation for complex tasks, like problem solving. Under the DCM, people indeed invest effort in a goal-driven manner. However, investing more time leads to increasing compromise on the goal, resulting in negative time-judgment correlations. Experiment 1 exposed that with word-pair memorization, negative correlations are found only with minimal fluency and difficulty variability, whereas in problem solving, they are found consistently. As predicted, manipulations of low incentives (Experiment 2) and time pressure (Experiment 3) in problem solving revealed greater compromise as more time was invested in a problem. Although intermediate confidence ratings rose during the solving process, the result was negative time-confidence correlations (Experiments 3, 4, and 5), and this was not eliminated by the opportunity to respond "don't know" (Experiments 4 and 5). The results suggest that negative time-judgment correlations in complex tasks stem from top-down regulatory processes with a criterion that diminishes with invested time.

Research in metacognition (Koriat, Ma'ayan, & Nussinson, 2006) suggests bidirectional links between monitoring and control during learning: When self-regulation is goal-driven, monitoring affects control so that increased study time (ST) enhances judgments of learning (JOLs). However, when self-regulation is data-driven, JOLs are based on the feedback from control, and therefore JOLs decrease with ST under the heuristic that ease of encoding is diagnostic of successful recall. Evidence for both types of relationships occurring within the same situation was found for adults. We examined the development of the ability to respond differentially to data-driven and goal-driven variation in ST within the same task. Children in Grades 5 and 6 exhibited a positive ST-JOL relationship for goal-driven regulation and a negative relationship for data-driven regulation but never in the same task. In contrast, the JOLs and recall of 9th graders and college students yielded differential cosensitivity to data-driven and goal-driven variation. The 5th and 6th graders also evidenced an adult-like pattern of JOLs and recall under a partitioning procedure that helped them in factoring the variation in ST due to data-driven and goal-driven variation in ST. The results are discussed in terms of the metacognitive sophistication needed for considering both types of variation simultaneously in making metacognitive judgments.

The original version of this article unfortunately contained a mistake in page 7, particularly in the second paragraph of “Procedure” section of Experiment 1: "… and only 0.3% of the answers were given in four seconds or more." The word "more" should be replaced with "less".

Although widely studied in other domains, relatively little is known about the metacognitive processes that monitor and control behaviour during reasoning and decision-making. In this paper, we examined the conditions under which two fluency cues are used to monitor initial reasoning: answer fluency, or the speed with which the initial, intuitive answer is produced (Thompson, Prowse Turner, & Pennycook, 2011), and perceptual fluency, or the ease with which problems can be read (Alter, Oppenheimer, Epley, & Eyre, 2007). The first two experiments demonstrated that answer fluency reliably predicted Feeling of Rightness (FOR) judgments to conditional inferences and base rate problems, which subsequently predicted the amount of deliberate processing as measured by thinking time and answer changes; answer fluency also predicted retrospective confidence judgments (Experiment 3b). Moreover, the effect of answer fluency on reasoning was independent from the effect of perceptual fluency, establishing that these are empirically independent constructs. In five experiments with a variety of reasoning problems similar to those of Alter et al. (2007), we found no effect of perceptual fluency on FOR, retrospective confidence or accuracy; however, we did observe that participants spent more time thinking about hard to read stimuli, although this additional time did not result in answer changes. In our final two experiments, we found that perceptual disfluency increased accuracy on the CRT (Frederick, 2005), but only amongst participants of high cognitive ability. As Alter et al.’s samples were gathered from prestigious universities, collectively, the data to this point suggest that perceptual fluency prompts additional processing in general, but this processing may results in higher accuracy only for the most cognitively able.

Previous studies in the domain of metacomprehension judgments have primarily used expository texts. When these texts include illustrations, even uninformative ones, people were found to judge that they understand their content better. The present study aimed to delineate the metacognitive processes involved in understanding problem solutions — a text type often perceived as allowing reliable judgments regarding understanding, and was not previously considered from a metacognitive perspective. Undergraduate students faced difficult problems. They then studied solution explanations with or without uninformative illustrations and provided judgments of comprehension (JCOMPs). Learning was assessed by application to near-transfer problems in an open-book test format. As expected, JCOMPs were polarized — they tended to reflect good or poor understanding. Yet, JCOMPs were higher for the illustrated solutions and even high certainty did not ensure resistance to this effect. Moreover, success in the transfer problems was lower in the presence of illustrations, demonstrating a bias stronger than that found with expository texts. Previous studies have suggested that weak learners are especially prone to being misled by superficial cues. In the present study, matching the difficulty of the task to the ability of the target population revealed that even highly able participants were not immune to misleading cues. The study extends previous findings regarding potential detrimental effects of illustrations and highlights aspects of the metacomprehension process that have not been considered before.

People often attribute their reluctance to study texts on screen to technology-related factors rooted in hardware or software. However, previous studies have pointed to screen inferiority in the metacognitive regulation of learning. The study examined the effects of time pressure on learning texts on screen relative to paper among undergraduates who report only moderate paper preference. In Experiment 1, test scores on screen were lower than on paper under time pressure, with no difference under free regulation. In Experiment 2 the time condition was manipulated within participants to include time pressure, free regulation, and an interrupted condition where study was unexpectedly stopped after the time allotted under time pressure. No media effects were found under the interrupted study condition, although technology-related barriers should have taken their effect also in this condition. Paper learners who preferred this learning medium improved their scores when the time constraints were known in advance. No such adaptation was found on screen regardless of the medium preference. Beyond that, paper learning was more efficient and self-assessments of knowledge were better calibrated under most conditions. The results reinforce the inferiority of self-regulation of learning on screen and argue against technology-related factors as the main reason for this.

Confidence in answers is known to be sensitive to the fluency with which answers come to mind. One aspect of fluency is response latency. Latency is often a valid cue for accuracy, showing an inverse relationship with both accuracy rates and confidence. The present study examined the independent latency-confidence association in problem-solving tasks. The tasks were ecologically valid situations in which latency showed no validity, moderate validity, and high validity as a predictor of accuracy. In Experiment 1, misleading problems, which often elicit initial wrong solutions, were answered in open-ended and multiple-choice test formats. Under the open-ended test format, latency was absolutely not valid in predicting accuracy: Quickly and slowly provided solutions had a similar chance of being correct. Under the multiple-choice test format, latency predicted accuracy better. In Experiment 2, nonmisleading problems were used; here, latency was highly valid in predicting accuracy. A breakdown into correct and incorrect solutions allowed examination of the independent latency-confidence relationship when latency necessarily had no validity in predicting accuracy. In all conditions, regardless of latency's validity in predicting accuracy, confidence was persistently sensitive to latency: The participants were more confident in solutions provided quickly than in those that involved lengthy thinking. The study suggests that the reliability of the latency-confidence association in problem solving depends on the strength of the inverse relationship between latency and accuracy in the particular task.

Recent work on metacognition indicates that monitoring is sometimes based itself on the feedback from control operations. Evidence for this pattern has not only been shown in adults but also in elementary schoolchildren. To explore whether this finding can be generalized to a wide range of age groups, 160 participants from first to eighth grade participated in a study based on a self-paced study time (ST) allocation paradigm. In contrast to previous studies, picture pairs instead of word pairs were used as stimuli to compensate for reduced reading skills in younger participants. Actual ST and judgments of learning (JOLs) made at the end of each study trial were used as core variables. The results are in line with previous findings, in that children's JOLs decreased with increasing ST, suggesting that JOLs were based on the memorizing effort heuristic that easily learned items are more likely to be remembered. Weaker inverse relationship between JOLs and ST was found for the younger children. Overall, these results underline the importance of mnemonic cues in shaping metacognitive feelings not only in adults but also in older children and expose a developmental trend in their use along childhood.

Research with adults indicates that confidence in the correctness of an answer decreases as a function of the amount of time it takes to reach that answer, suggesting that people use response latency as a mnemonic cue for subjective confidence. Experiment 1 extended investigation to 2nd, 3rd and 5th graders. When children chose the answer to general knowledge questions, their confidence in the answer was inversely related to choice latency. However, the strength of the relationship increased with grade, suggesting increased reliance with age on the feedback from task performance. The validity of latency as a cue for the accuracy of the answer also increased with age, possibly contributing to the observed age increase in the extent to which confidence judgment discriminated between correct and wrong answers. Whereas these results illustrate the dependence of metacognitive monitoring on the feedback from control operations, Experiments 2 and 3 examined the idea that control-based monitoring affects subsequent control operations. When children were free to choose which answers to volunteer under a payoff schedule that emphasized accuracy, they tended to volunteer high-confidence answers more than low-confidence answers (Experiment 2) and more short-latency answers than long-latency answers (Experiment 3). The latter tendency was again stronger for older than for younger children. The results are discussed in terms of the intricate relationships between monitoring and control processes.

When answering questions from memory, respondents strategically control the precision or coarseness of their answers. This grain control process is guided by 2 countervailing aims: to be informative and to be correct. Previously, M. Goldsmith, A. Koriat, and A. Weinberg Eliezer (2002) proposed a satisfying model in which respondents provide the most precise answer that passes a minimum-confidence report criterion. Pointing to social-pragmatic considerations, the present research shows the need to incorporate a minimum-informativeness criterion as well. Unlike its predecessor, the revised, "dual-criterion" model implies a distinction between 2 theoretical knowledge states: Under moderate-to-high levels of satisfying knowledge, a grain size can be found that jointly satisfies both criteria--confidence and informativeness. In contrast, under lower levels of unsatisfying knowledge, the 2 criteria conflict--one cannot be satisfied without violating the other. In support of the model, respondents often violated the confidence criterion in deference to the informativeness criterion, particularly when answering under low knowledge, despite having full control over grain size. Results also suggest a key role for the "don't know" response which, when available, can be used preferentially to circumvent the criterion conflict.

The same event that appeared unpredictable in foresight can be judged as predictable in hindsight. Hindsight bias clouds judgments in all areas of life, including legal decisions, medical diagnoses, consumer satisfaction, sporting events, and election outcomes. We discuss three theoretical constructs related to hindsight bias: memory, reconstruction bias, and motivation. Attempts to recall foresight knowledge fail because newly acquired knowledge affects memory either directly or indirectly by biasing attempts to reconstruct foresight knowledge. On a metacognitive level, overconfidence and surprise contribute to hindsight bias. Overconfidence in knowledge increases hindsight bias whereas a well-calibrated confidence reduces hindsight bias. Motivational factors also contribute to hindsight bias by making positive and negative outcomes appear more or less likely, depending on a variety of factors. We review hindsight bias theories and discuss three exciting directions for future research.

In this reply, we provide an analysis of Alter et al. (2013) response to our earlier paper ( Thompson et al., 2013 ). In that paper, we reported difficulty in replicating Alter, Oppenheimer, Epley, and Eyre’s (2007) main finding, namely that a sense of disfluency produced by making stimuli difficult to perceive, increased accuracy on a variety of reasoning tasks. Alter, Oppenheimer, and Epley (2013) argue that we misunderstood the meaning of accuracy on these tasks, a claim that we reject. We argue and provide evidence that the tasks were not too difficult for our populations (such that no amount of “metacognitive unease” would promote correct responding) and point out that in many cases performance on our tasks was well above chance or on a par with Alter et al.’s (2007) participants. Finally, we reiterate our claim that the distinction between answer fluency (the ease with which an answer comes to mind) and perceptual fluency (the ease with which a problem can be read) is genuine, and argue that Thompson et al. (2013) provided evidence that these are distinct factors that have different downstream effects on cognitive processes.

In self-paced learning, when the regulation of study effort is goal driven (e.g., allocated to different items according to their relative importance), judgments of learning (JOLs) increase with study time. When regulation is data driven (e.g., determined by the ease of committing the item to memory), JOLs decrease with study time (Koriat, Ma'ayan, & Nussinson, 2006). We induced learners to interpret differences in their study time (Experiment 1) or in another learner's study time (Experiment 2) as reflecting either differences in data-driven regulation or differences in goal-driven regulation. This manipulation was found to moderate the relationship of both study time and rated effort to JOLs. The results were seen to support the idea that JOLs are based on study effort but the effects of experienced effort are mediated by an attribution that intervenes between the metacognitive regulation of effort and the monitoring of one's learning. The results invite an attributional theoretical framework that encompasses both data-driven and goal-driven regulation and incorporates the option of attributing experienced effort to either or both of the 2 types of regulation.

In this reply, we provide an analysis of Alter et al. (2013) response to our earlier paper (Thompson et al., 2013). In that paper, we reported difficulty in replicating Alter, Oppenheimer, Epley, and Eyre's (2007) main finding, namely that a sense of disfluency produced by making stimuli difficult to perceive, increased accuracy on a variety of reasoning tasks. Alter, Oppenheimer, and Epley (2013) argue that we misunderstood the meaning of accuracy on these tasks, a claim that we reject. We argue and provide evidence that the tasks were not too difficult for our populations (such that no amount of "metacognitive unease" would promote correct responding) and point out that in many cases performance on our tasks was well above chance or on a par with Alter et al.'s (2007) participants. Finally, we reiterate our claim that the distinction between answer fluency (the ease with which an answer comes to mind) and perceptual fluency (the ease with which a problem can be read) is genuine, and argue that Thompson et al. (2013) provided evidence that these are distinct factors that have different downstream effects on cognitive processes.