Search

Open Learner Models (OLM) show the learner model to users to assist their self-regulated learning by, for example, helping prompt reflection, facilitating planning and supporting navigation. OLMs can show different levels of detail of the underlying learner model, and can also structure the information differently. As a result, a trade-off may exist between the potential for better support for learning and the complexity of the information shown. This paper investigates students' perceptions about whether offering more and richer information in an OLM will result in more effective support for their self-regulated learning. In a first study, questionnaire responses relating to designs for six visualisations of varying complexity led to the implementation of three variations on one of the designs. A second controlled study involved students interacting with these variations. The study revealed that the most useful variation for searching for suitable learning material was a visualisation combining a basic coloured grid, an extended bar chart-like visualisation indicating related concepts, and a learning gauge.

Open Learner Models (OLM) show the learner model to users to assist their self-regulated learning by, for example, helping prompt reflection, facilitating planning and supporting navigation. OLMs can show different levels of detail of the underlying learner model, and can also structure the information differently. As a result, a trade-off may exist between the potential for better support for learning and the complexity of the information shown. This paper investigates students' perceptions about whether offering more and richer information in an OLM will result in more effective support for their self-regulated learning. In a first study, questionnaire responses relating to designs for six visualisations of varying complexity led to the implementation of three variations on one of the designs. A second controlled study involved students interacting with these variations. The study revealed that the most useful variation for searching for suitable learning material was a visualisation combining a basic coloured grid, an extended bar chart-like visualisation indicating related concepts, and a learning gauge.

How much information about meaning is contained in the statistical structure of the environment? LSA is a theoretical and practical tool that is challenging previous notions about what is contained in the statistical structure of the environment. This paper examines what kind of category knowledge can be obtained from the environment using LSA. In particular, two experiments are conducted with LSA to test what kind of category structure it embodies. LSA ratings about the relatedness of categories to their properties are compared with human judgments regarding the centrality of properties to the categories. LSA is found to capture aspects of property centrality for some object and event categories. However, it is found to only capture those aspects related to typicality: how often do members of the category have that property? LSA fails to capture other aspects of centrality that can be found in human category judgments. Thus, it appears that humans do bring other constraints to bear in shaping their categories.

Previous research on scientific reasoning has shown that it involves a diverse set of skills. Yet, little is known about generality of those skills, an important issue to theories of expertise and to attempts to automate scientific reasoning skills. We present a study examining what kinds of skills psychologists actually use in designing and interpreting experiments. The results suggest: 1) that psychologists use many domain-general skills in their experimentation; 2) that bright and motivated undergraduates are missing many of these skills; 3) some domain-general skills are not specific to only scientists; and 4) some domain-specific skills can be acquired with minimal domain-experience.

The field of cognitive science is inherently multi?disciplinary. However, it is unclear to what extent truly interdisciplinary work occurs in cognitive science. That is, is cognitive science merely a collection of researchers from different disciplines working separately on commo n problems? Data gathered from a recent cognitive science conference are presented. Interestingly, a significant proportion of interdisciplinary collaborations were found. Analyses were also conducted on the impact of same vs. different backgrounds on the structure of collaborations, and It was found that interdisciplinary collaborations involved more equally distributed contributions among the authors than did intradisciplinary collaborations.

An extension of Klahr and Dunbar's (1988) Dual space model of scientific discovery is presented. W e propose that, in addition to search in an experiment space and a hypothesis space, scientific discovery involves search in two additional spaces; the space of data representations and the space of experimental paradigms. That is, discoveries often involve developing new terms and adding new features to descriptions of the data, and the also often involve developing new kinds of experimental procedures. The 4-space model was motivated by the analysis of human performance in a discovery microworld. A brief description of the data is presented. In addition to the general 4-space framework, a description of the component processes involved in each of the four search spaces is also presented

The purpose of this study was to characterize the ways in which psychologists address research hypothesis risk in academic articles, and to support undergraduates in learning to write about such risk using argument diagramming prewriting activities. First, 90 articles recently published in top social, developmental, and cognitive psychology journals were examined for their presentation of research hypothesis ‘risk’ – an element of the intellectual merit of a research study denoting the novelty and importance of the study being conducted. Second, an experimental study was conducted involving 82 students in undergraduate research methods classes. They were assigned to either argument diagram or traditional instruction conditions. Research reports were coded for explicit discussion of risk. Students using argument diagramming were significantly more likely to write about risk when compared to matched classes given no diagramming support

Symbolic and non-symbolic number representations are thought to share common neural substrates. However, recent studies have shown that the two numerical systems are more distinct than previously thought. These disparate findings may be explained by the use of sequential presentations of symbolic and non-symbolic quantities, the use of magnitude-reliant tasks, or the use of limited number ranges. We investigated whether adults integrate symbolic and non-symbolic numerical information during a non-magnitude-based task in which symbolic and non-symbolic double-digit numerical information is shown simultaneously. Participants viewed images in which symbolic numerals or letter pairs were superimposed on non-symbolic numerical stimuli and were asked to determine whether the text was a numeral or letter, ignoring the dots. After perceptual biases were taken into account, participants were more accurate and faster in their judgments when symbolic and non-symbolic information matched than when information mismatched, suggesting that adults can integrate symbolic and non-symbolic numerical information

Discourse comprehension processes attempt to produce an elaborate and well-connected representation in the reader’s mind. A common network of regions including the angular gyrus, posterior cingulate, and dorsal frontal cortex appears to be involved in constructing coherent representations in a variety of tasks including social cognition tasks, narrative comprehension, and expository text comprehension. Reading strategies that require the construction of explicit inferences are used in the present research to examine how this coherence network interacts with other brain regions. A psychophysiological interaction analysis was used to examine regions showing changed functional connectivity with this coherence network when participants were engaged in either a non-inferencing reading strategy, paraphrasing, or a strategy requiring coherence-building inferences, self-explanation. Results of the analysis show that the coherence network increases in functional connectivity with a cognitive control network that may be specialized for the manipulation of semantic representations and the construction of new relations among these representations.

Discourse comprehension processes attempt to produce an elaborate and well-connected representation in the reader’s mind. A common network of regions including the angular gyrus, posterior cingulate, and dorsal frontal cortex appears to be involved in constructing coherent representations in a variety of tasks including social cognition tasks, narrative comprehension, and expository text comprehension. Reading strategies that require the construction of explicit inferences are used in the present research to examine how this coherence network interacts with other brain regions. A psychophysiological interaction analysis was used to examine regions showing changed functional connectivity with this coherence network when participants were engaged in either a non-inferencing reading strategy, paraphrasing, or a strategy requiring coherence-building inferences, self-explanation. Results of the analysis show that the coherence network increases in functional connectivity with a cognitive control network that may be specialized for the manipulation of semantic representations and the construction of new relations among these representations.

Other-generated hypotheses are often considered easier to test than self-generated hyptitheses. T o determine the precise effects of other-generated hypotheses, w e propose three kinds of effects and describe a study designed to test for these effects of hypothesis source. T h e three kinds of effects considered are: (i) hypothesis plausibility changes, (ii) skepticism changes, and (iii) process changes. Forty-two undergraduate subjects were given a microworld discovery task called Milktruck. Subjects either had to generate their o w n initial hypothesis or were given the most frequently generated hypothesis. It w a s found that the other- generated hypothesis lead to m o r e thorough investigation of hypotheses resulting in a decrease in false terminations with incorrect solutions. T h e results suggested these effects were caused by an increjise in skepticism rather than changes in hypothesis plausibility or process changes.

Previous psychological research about scientific discovery has often focused on subjects' heuristics for discovering simple concepts with one relevant dimension or a few relevant dimensions with simple two-way interactions. This p^er presents results from an experiment in which subjects had to discover a concept involving complex three-way interactions on a multi-valued output by running experiments in a computerized microworld. Twenty-two C M U undergraduates attempted the task, of which sixteen succeeded, in an average of 85 minutes. The analyses focus on three strategies used to regulate task complexity. First, subjects preferred depth-first to breadth-first search, with successful subjects regulating the number of features varied from experiment to experiment most effectively. Second, subjects systematically regulated the length of their experiments. Third, a new explicit search heuristic (Put Upon Stack Heuristic) used by successful subjects is described.

Chapter in: The Journey From Child to Scientist: Integrating Cognitive Development and the Education Sciences

Chapter in Handbook of the Psychology of Science

This paper presents an interactive analytic tool for educational peer-review analysis. It employs data visualization at multiple levels of granularity, and provides automated analytic support using clustering and natural language processing. This tool helps instructors discover interesting patterns in writing performance that are reflected through peer reviews.

Chapter in: The Journey From Child to Scientist: Integrating Cognitive Development and the Education Sciences

Chapter in Handbook of the Psychology of Science

This paper presents an interactive analytic tool for educational peer-review analysis. It employs data visualization at multiple levels of granularity, and provides automated analytic support using clustering and natural language processing. This tool helps instructors discover interesting patterns in writing performance that are reflected through peer reviews.

Chapter in Psychology of Learning and Motivation (Vol. 53)

Recruiting and retaining highly qualified physics and physical science teachers is critical for maintaining America's global competitiveness. Unfortunately, many high school and middle school teachers are asked to teach science subjects they do not feel comfortable teaching and are not provided adequate guidance and support. Moreover, teachers often lack adequate pedagogical content knowledge to teach science effectively. Here, we discuss the development, implementation, and assessment of a course for science and engineering undergraduates designed to increase awareness and help them develop an interest and a deeper appreciation of the intellectual demands of physics teaching. The course focused on increasing student enthusiasm and confidence in teaching by providing well supported teaching opportunities and exposure to physics education research. The course assessment methods include 1) pre-test and post-test measures of attitude and expectations about science teaching, 2) self and peer evaluation of student teaching, 3) content-based pre-tests and post-tests given to students who received instruction from the student teachers, and 4) audio-taped focus group discussions in the absence of the instructor and TA to evaluate student perspectives on different aspects of the course and its impact.

We believe that providing assessment on students ’ reviewing performance will enable students to improve the quality of their peer reviews. We focus on assessing one particular aspect of the textual feedback contained in a peer review – the presence or absence of problem localization; feedback containing problem localization has been shown to be associated with increased understanding and implementation of the feedback. While in prior work we demonstrated the feasibility of learning to predict problem localization using linguistic features automatically extracted from textual feedback, we hypothesize that inter-annotator disagreement on labeling problem localization might impact both the accuracy and the content of the predictive models. To test this hypothesis, we compare the use of feedback examples where problem localization is labeled with differing levels of annotator agreement, for both training and testing our models. Our results show that when models are trained and tested using only feedback where annotators agree on problem localization, the models both perform with high accuracy, and contain rules involving just two simple linguistic features. In contrast, when training and testing using feedback examples where annotators both agree and disagree, the model performance slightly drops, but the learned rules capture more subtle patterns of problem localization. Keywords problem localization in text comments, data mining of peer reviews, inter-annotator agreement, natural languag

Recruiting and retaining highly qualified physics and physical science teachers is critical for maintaining America's global competitiveness. Unfortunately, many high school and middle school teachers are asked to teach science subjects they do not feel comfortable teaching and are not provided adequate guidance and support. Moreover, teachers often lack adequate pedagogical content knowledge to teach science effectively. Here, we discuss the development, implementation, and assessment of a course for science and engineering undergraduates designed to increase awareness and help them develop an interest and a deeper appreciation of the intellectual demands of physics teaching. The course focused on increasing student enthusiasm and confidence in teaching by providing well supported teaching opportunities and exposure to physics education research. The course assessment methods include 1) pre-test and post-test measures of attitude and expectations about science teaching, 2) self and peer evaluation of student teaching, 3) content-based pre-tests and post-tests given to students who received instruction from the student teachers, and 4) audio-taped focus group discussions in the absence of the instructor and TA to evaluate student perspectives on different aspects of the course and its impact.

Chapter in Psychology of Learning and Motivation (Vol. 53)

We believe that providing assessment on students ’ reviewing performance will enable students to improve the quality of their peer reviews. We focus on assessing one particular aspect of the textual feedback contained in a peer review – the presence or absence of problem localization; feedback containing problem localization has been shown to be associated with increased understanding and implementation of the feedback. While in prior work we demonstrated the feasibility of learning to predict problem localization using linguistic features automatically extracted from textual feedback, we hypothesize that inter-annotator disagreement on labeling problem localization might impact both the accuracy and the content of the predictive models. To test this hypothesis, we compare the use of feedback examples where problem localization is labeled with differing levels of annotator agreement, for both training and testing our models. Our results show that when models are trained and tested using only feedback where annotators agree on problem localization, the models both perform with high accuracy, and contain rules involving just two simple linguistic features. In contrast, when training and testing using feedback examples where annotators both agree and disagree, the model performance slightly drops, but the learned rules capture more subtle patterns of problem localization. Keywords problem localization in text comments, data mining of peer reviews, inter-annotator agreement, natural languag

We believe that providing assessment on students ’ reviewing performance will enable students to improve the quality of their peer reviews. We focus on assessing one particular aspect of the textual feedback contained in a peer review – the presence or absence of problem localization; feedback containing problem localization has been shown to be associated with increased understanding and implementation of the feedback. While in prior work we demonstrated the feasibility of learning to predict problem localization using linguistic features automatically extracted from textual feedback, we hypothesize that inter-annotator disagreement on labeling problem localization might impact both the accuracy and the content of the predictive models. To test this hypothesis, we compare the use of feedback examples where problem localization is labeled with differing levels of annotator agreement, for both training and testing our models. Our results show that when models are trained and tested using only feedback where annotators agree on problem localization, the models both perform with high accuracy, and contain rules involving just two simple linguistic features. In contrast, when training and testing using feedback examples where annotators both agree and disagree, the model performance slightly drops, but the learned rules capture more subtle patterns of problem localization. Keywords problem localization in text comments, data mining of peer reviews, inter-annotator agreement, natural languag

Teaching mathematics in a technology classroom requires more than simply using mathematics with technology. It requires designing the lesson to focus, motivate, and highlight the mathematics in a meaningful way.

Chapter in: Psychology of Classroom Learning: An Encyclopedia.

Chapter in: Psychology of Classroom Learning: An Encyclopedia

Chapter in: Psychology of Classroom Learning: An Encyclopedia.

Chapter in: Psychology of Classroom Learning: An Encyclopedia

Teaching mathematics in a technology classroom requires more than simply using mathematics with technology. It requires designing the lesson to focus, motivate, and highlight the mathematics in a meaningful way.

Various analogies have been applied to educational research, the most prominent current one being medical clinical trials. What about the design and development of educational materials? There are many design principles being developed for educational materials, but the process of design is much less codified or systematic. By contrast, engineering as a discipline has created very systematic processes for doing design work, whether the design of products, processes, systems, or services, similar to the systematic scientific processes that are applied across the sciences. This paper overviews major findings from the study of engineering design practices and then presents an analysis of the extent to which engineering design processes may and may not be usefully applied to the process of educational design.

Various analogies have been applied to educational research, the most prominent current one being medical clinical trials. What about the design and development of educational materials? There are many design principles being developed for educational materials, but the process of design is much less codified or systematic. By contrast, engineering as a discipline has created very systematic processes for doing design work, whether the design of products, processes, systems, or services, similar to the systematic scientific processes that are applied across the sciences. This paper overviews major findings from the study of engineering design practices and then presents an analysis of the extent to which engineering design processes may and may not be usefully applied to the process of educational design.

Imagine a meteorologist preparing a weather forecast. In addition to years of experience and a vast store of domain knowledge, the forecaster has access to satellite images, to computer generated weather models and programs to display them in a variety of ways, and to an assortment of special-purpose tools that provide additional task-relevant data. There is no shortage of data, yet despite this array of resources, the task remains very challenging. One source of complexity is the uncertainty inherent in these data. To complicate matters further, the uncertainty in the data is not explicitly represented; rather, the visualizations indicate that the data are exactly as they appear. The visualizations thus invite the forecaster to map uncertain data to certain values, yet to do so would most likely lead to erroneous predictions. This example illustrates the basic question we investigate in this paper: how do people, especially experts, deal with uncertainty in highly spatial domains, when the data are inherently uncertain but the tools actually display very little uncertainty? We first examine how uncertainty affects operations in three representative domains, submarine operations (military), meteorology (geoscience), and fMRI research (scientific visualization), in which dealing with uncertainty is a critical component of the task. We then investigate how experts in two of these domains, meteorology and fMRI, manage uncertainty as they perform problem-solving activities and make decisions as part of their regular task performance., Prepared in collaboration with the University of Pittsburgh. Published in Thinking with Data, edited by Lovett and Shah, Psychology Press, 2007. ISBN 978-0-8058-5422-0.

How do problem solvers represent visual-spatial information in complex problem solving tasks? This paper explores the predictions of symbolic computation, embodied problem solving and a neurocomputational theory for what factors influence internal representation choices. Across two studies, data are collected from experts and novices in three different, complex visual-spatial problem-solving domains (weather forecasting, submarine target motion analysis, and fMRI data analysis). Internal spatial representations are coded from spontaneous gestures made during cued-recall summaries of problem solving activities. Analyses of domain differences, expertise differences, and changes over time with problem solving suggest that neurocomputational constraints play a larger role than the nature of the visual input or the nature of the underlying real world being examined through problem solving, especially for expert problem solvers. The particular neurocomptuational feature that was found to drive internal representation choice is the required spatial precision of the main goals of problem solving., Prepared in collaboration with University of Pittsburgh, grant no's. N00014-02-1-0113 and N00014-03-1-0061.

This research tested the Adaptive Strategies Model (ASM; Lemaire & Siegler, 1995) of expertise in the ill-defined domain of leadership. More specifically, we examined expert/novice differences in all components of the ASM: strategy existence, strategy choice, strategy base-rate, and strategy execution. In Experiment 1 Leadership Scenarios elicited free-text responses from undergraduates (novices), ROTC Cadets (intermediates), and US Army Platoon Leaders (experts). Each response was coded into one of ten underlying Leadership Strategies thought to underlie each response, resulting in patterns of individual strategy use. Experiment 2 used a new group of experts to gather ratings of the execution accuracy of responses from Experiment 1. The results show that the ASM is able to discern expert/novice differences in strategy choice, strategy base-rate, and strategy execution. As leaders progress from novice to expert, they 1) use multiple strategies across various scenarios, 2) develop the ability to make optimal choices about when and where to use particular strategies, 3) develop an increased sensitivity to each different strategy's base rate of success in the environment, and 4) develop the ability to execute strategies more accurately. The training of ill-defined skills, such as leadership, may be improved by focusing on the four components of ASM.