Search

Online group debates hosted in asynchronous threaded discussions can facilitate critical thinking between discussants (and increase deeper understanding of complex problems) by eliminating the need for turn-taking while formulating and presenting premises to support and challenge claims. Yet to be determined is to what extent does the current number of posts (premises from teammates, premises from the opposing team, supportive replies to premises, and opposing replies to premises) induce production blocking to disrupt the generation of new premises. Statistical discourse analysis was conducted on 1554 postings from online debates produced by four cohorts of students enrolled in an online graduate-level course. The resulting stochastic model revealed that: (a) the number of premises posted by the other team, premises posted by teammates, and supportive replies from teammates were associated with the largest to smallest drop in the likelihood of posting a new premise, respectively; and (b) the number of oppositional replies from the other team was associated with an increased likelihood of posting a new premise. These findings provide evidence to support the use of specific strategies for structuring and sequencing the argumentation task to generate larger numbers of premises to achieve a deeper and more thorough analysis of problems and claims.

Despite the last 40 years of research showing that computer-aided diagramming tools improve student learning, very little research reveals the cognitive processes that explain why and how diagramming tools and specific features of the tools affect learning. This study developed a tool that graduate students used to diagram and analyze arguments as the tool mined each students’ actions. The mined data was used to develop algorithms to operationalize and measure the use of backward, forward, breadth, depth-first reasoning. Regression models were then compared to identify which algorithm produced measures that best predicted diagram scores, and to determine the combined and relative impact of each reasoning process on diagram scores. The findings show that observing the placement and location of the first five nodes moved and positioned on screen in relation to the location of the previously moved node provides sufficient data to generate backward/forward and breadth/depth-first ratio scores that predict diagram scores, while individual frequency counts of each of the four processes do not predict scores. The best-fit regression model using the ratio scores show that students using more backward and depth-first processing construct diagrams with higher scores – scores based on criteria that gauge the depth of analysis and not just the number of correct diagram links. This study presents new tools, methods, and new lines of inquiry to advance research on ways to integrate learning analytics into computer-aided diagramming tools.

We examined how social antecedents impact students’ use of explanations versus evidence to justify arguments using statistical discourse analysis on 2028 postings from 87 graduate students in five courses, each participating in four online debates. The results show that students overall were much more likely to justify arguments with explanations than with evidence. Explanations were more likely than evidence to be used in postings by women, when students posted responses to messages that conveyed greater social proximity (using he/she/they and using you instead of we) or directed attention (there), when making posts in early parts of a discussion thread and in the opening argument. Evidence was more likely to be used when responding to messages from men and when making posts towards the end of each discussion thread.

According to Gagne, instruction should follow the prescribed nine events of instruction, but that the sequence need not be absolute and that not all events are necessary. The purpose of this study was to determine to what extent are the prescribed sequence implemented in practice, and how might variations in event sequences affect learner satisfaction. Using sequential analysis, this study identified patterns in instructional event sequences observed in iTune audio podcasts with low versus high user satisfaction ratings. The analysis revealed that collectively the audio podcasts followed the event sequence prescribed by Gagne. However, the high-rated podcasts were more likely to transition from objectives straight to information presentation (skipping stimulating recall) and from information presentation to practice (skipping guidance). If learners were using the podcasts to support just-in-time learning and were aware that they could return to the podcasts at any time in the future, there was no need to stimulate recall and receive guidance to improve long-term memory and recall. These findings suggest that event sequences should be modified to accommodate different instructional contexts in order to increase learner satisfaction and efficiency. This study and its findings also serve to illustrate ways to apply sequential analysis to conduct further investigations on ways to vary instructional event sequences to optimize both process and learning outcomes.

Given that grammatical and spelling errors have been found to influence perceived competence and credibility in written communication, this study examined how a student’s grammar and spelling errors affect how other students respond to the student’s postings in four online debates hosted in asynchronous threaded discussions. Message-response exchanges were sequentially analyzed to identify patterns in students’ replies to arguments and challenges with counter-challenges, explanations, and evidentiary support posted by students that exhibited low versus high number of grammatical and spelling errors. Although no causal inferences can be drawn from this study, the findings nevertheless suggests that: (a) arguments posted by high-error students are more likely to be challenged than arguments posted by low-error students; (b) exchanges between high-error students can amplify the effects of grammar/spelling errors; and (c) higher levels of argumentation can be achieved by placing students into groups that are heterogeneous in writing skills in general. The findings and methods used in this study lay the groundwork for further research on strategies for managing individual differences in students’ grammar and spelling (and other student behaviors in general) and increasing the level of critical discourse in online discussions.

Because prior beliefs can affect how people argue and respond to controversial claims, in this study we examined how patterns in students’ responses posted in online debates were associated with students’ initial positions on given claims prior to debate. Students that held initial opposing positions to the claim exhibited the tendency to respond to challenges from the opposition with explanations and to respond to explanations from the opposition with challenges. This response pattern was not observed among students with initial supporting and neutral positions. These findings suggest that manipulating group composition based on prior beliefs and manipulating the direction and phrasing of the claim under debate may help to elicit more frequent exchange of opposing viewpoints and raise the level of critical discourse.

The purpose of this study was to examine the learning process exhibited in restrained online debates in terms of to what extent each of Bloom’s six levels of cognitive learning were exhibited among four types of message (argument, critique, evidence, and explanation). Thirty-three graduate students enrolled in an online entry-level course in distance learning participated in this study. The threaded discussion postings from four structured online debates were analyzed. The results indicated that five of the six levels of cognitive learning (except for knowledge learning) were achieved to a certain extent among the four types of postings. Chi square tests indicated that higher levels of learning were most likely to be exhibited in critique and argument postings. Message–response exchanges ending with critiques or starting with argument messages were most likely to elicit higher levels responses. The results were discussed in light of better understanding of student learning and implications for instructional designs of discussion topics and restraints.

This study examined some of the methodological approaches used by students to construct causal maps in order to determine which approaches help students understand the underlying causes and causal mechanisms in a complex system. This study tested the relationship between causal understanding (ratio of root causes correctly/incorrectly identified, number of correctly identified root-cause links explaining how root causes directly/indirectly impact final outcomes) and three attributes observed in students’ causal maps (total links, temporal flow, lateral position of final outcome) that students produced before and after online discussions on noted similarities and differences between students’ causal maps. The findings suggest that: (a) causal understanding can be adversely affected if students are instructed before group discussion to temporally sequence nodes to flow from left to right and to position the outcome node farther away from the left edge of the map relative to other nodes in the map; (b) causal understanding following group discussion can be increased by instructing students to minimize the number of causal links and create a map with temporally flow; (c) promoting temporal flow following discussion may be the most effective means of helping students to identify root causes; and (d) instructing students to minimize the number of links following discussion may be the most effective means to helping students explain root causes directly/indirectly impact outcomes. These findings provide insights on what processes and constraints can be formalized and integrated into causal mapping software when used as an instructional and assessment tool.

Verbal and nonverbal communication is a critical mediator of performance in team sports and yet there is little extant research in sports that involves direct measures of communication. Our study explored communication within NCAA Division I female tennis doubles teams. Video and audio recordings of players during doubles tennis matches captured the communications that took place between and during points. These recordings were coded and sequential analysis computed using the Discussion Analysis Tool software (Jeong, 2003). Results indicated that most communications were emotional (i.e., > 50%) or action statements (i.e., > 25%). Winning teams exhibited significantly different communication sequences than losing teams. In particular, winning teams had a more homogeneous model of communication, which perhaps makes message interpretation more reliable. Finally, winning teams exchanged twice as many messages as losing teams.

In asynchronous threaded discussions, messages posted near the end of the week provide less time for students to critically examine and respond to ideas presented in the messages than messages posted early in the week. This study examined how the day in which messages are posted (early, midweek and weekend) in computer-supported collaborative argumentation affect the number of responses elicited by arguments and challenges, and how its effects differ across the four types of exchanges (argument–challenge, challenge–counterchallenge, challenge–explain, challenge–evidence) that serve as indicators of critical discourse. This study found that the day of posting had a significant effect on the number of responses elicited per message, with the greatest to smallest effect on argument–challenge, challenge–counterchallenge, challenge–explain and challenge–evidence exchanges respectively. These findings highlight some of the limitations of asynchronous discussions, and suggest alternative ways to implement and design discussion environments to promote higher levels of critical discourse. [ABSTRACT FROM AUTHOR]

This case study examined interaction patterns between men and women and the effects of the patterns on gender participation in online debates. Students labeled messages to identify each message as an argument, challenge, evidence, or explanation when posting messages to the debates. The results revealed no differences in the number of male and female challenges elicited by male versus female messages, number of male and female rebuttals (e.g., explanations and counterchallenges) elicited by male versus female challenges, and the frequency of challenges and explanations posted by men versus women. These findings suggest that gender differences in communication styles do not necessarily produce gender differences in response patterns and participation. Other factors such as male–female ratio and task structures can affect gender interaction patterns and gender participation. Nevertheless, this study demonstrated how interaction patterns can provide causal explanations for observed differences and similariti...

In this study we examined response patterns in exchanges between males and females and their effects on gender participation in five online debates. Students classified messages into arguments, evidence, critiques, and elaborations while posting messages to the debates to facilitate argumentation and the sequential analysis of message-response sequences. The findings revealed no differences in number of critiques posted in response to arguments because females were just as likely as males to critique messages from both males and females, and because females responded to males with critiques at a higher than expected frequency. Posthoc analysis revealed strong indications that females posted fewer rebuttals to the critiques of females than males, and males posted more rebuttals to the critiques of females than females. The methods used in this study illustrate a process-oriented approach to explain and predict gender differences in participation and serve as a framework for future research on gender participation, group interaction, and strategies for facilitating collaborative argumentation and problem solving.

This study examined the effects of pre-structuring discussion threads on group performance in computer-supported collaborative argumentation where students labeled their messages as arguments, challenges, supporting evidence, and explanations on a threaded discussion board. In the pre-structured group students were required to post supporting and opposing arguments to designated and separate discussion threads. In the control group no discussion threads were designated for posting supporting and opposing arguments. The mean number of challenges elicited per argument was 64% greater with pre-structured threads (ES = +.47). No significant differences were found in the mean number of counter-challenges, supporting evidence, and explanations posted in reply to challenges. The findings suggest that prestructured discussion threads can increase the frequency of argument→challenge exchanges needed to initiate critical discourse.

This study examined the effects of conversational language (e.g., asking questions, inviting replies, acknowledgments, referencing others by name, closing signatures, ‘I agree, but’, greetings, etc.) on the frequency and types of responses posted in reply to given types of messages (e.g., argument, evidence, critique, explanation), and how the resulting response patterns support and inhibit collaborative argumentation in asynchronous online discussions. Using event sequence analysis to analyze message-response exchanges in eight online group debates, this study found that (a) arguments elicited 41% more challenges when presented with more conversational language (effect size .32), (b) challenges with more conversational language elicited three to eight times more explanations (effect size .12 to .31), and (c) the number of supporting evidence elicited by challenges was not significantly different from challenges that used more versus less conversational language. Overall, these and other findings from exploratory post-hoc tests show that conversational language can help to produce patterns of interaction that foster high levels of critical discourse, and that some forms of conversational language are more effective in eliciting responses than others.

This paper proposes a set of methods and a framework for evaluating, modeling, and predicting group interactions in computer‐mediated communication. The method of sequential analysis is described along with specific software tools and techniques to facilitate the analysis of message–response sequences. In addition, the Dialogic Theory and its assumptions are presented to establish a theoretical framework and guide to using sequential analysis in computer‐mediated communication research. Step‐by‐step instructions are presented to illustrate how sequential analysis can be used to measure the way latent variables (e.g., message function, response latency, communication style) and exogenous variables (e.g., gender, discourse rules, context) affect how likely a message is to elicit a response, the types of responses elicited by the message, and whether or not the elicited sequence of responses (e.g., claim → challenge → explain) mirror the processes that support group decision‐making, problem‐solving, and lear...

The purpose of this study was to examine group interaction and critical thinking in online threaded discussions. The Discussion Analysis Tool (DAT) was used to identify patterns in interactions and...

A long-term instructional experiment assessed the cognitive consequences of reflective teaching practices, compared to inquiry teaching practices, on the acquisition and transfer of Logo computer programming for 2nd- and 5th-grade students. Inquiry teaching represented previous "best practices" in which teachers elicited predictions, asked leading questions, and assisted students when they encountered programming impasses. Reflective teaching modified inquiry teaching by explicit encouragement of a design stance where students assumed roles as potential designers of Logo as well as actual roles as the designers of their own programs for peer audiences. Other tools for reflection included writing summaries of their programming experiences, using programming templates, and working with "microworlds" that helped students objectify their experiences with Logo. Multiple levels (syntactic, semantic, schematic, strategic, and beliefs) of Logo knowledge were measured during and after instruction. Between-group di...

A growing body of research shows that deeper understanding of complex systems can be achieved when students construct causal maps to articulate, explore, and refine their understanding. However, to what extent is students’ maps an actual measure of their causal understanding versus the efficacy of different processes used by students to construct their map? To address this question, a set of methods was developed and implemented in this study to mine data from the causal mapping tool, jMAP; sequentially analyze the mined data; and construct transitional state diagrams to visualize, model, and compare students’ causal mapping behaviors to identify differences in action sequences performed by students that created maps of high versus low accuracy. The causal maps of 17 graduate students enrolled in an online course on collaborative learning at Florida State University (USA) were sequentially analyzed and used to illustrate how the proposed methods can be implemented to capture, analyze, model, and identify causal mapping and reasoning processes that produce deeper causal understanding. Identifying and validating the most effective processes will enable software developers to create causal mapping software that can standardize the map construction process, control for individual differences in mapping skills, enable students to produce causal maps that accurately reflect their causal understanding (and/or help students to produce more accurate causal maps), and thus enable instructors to use such mapping tools to conduct automated large-scale assessments of students’ causal understanding and systems thinking skills.

The national science standards call for increased focus on scientific argumentation in the classroom, and researchers have developed sophisticated online science learning environments to promote and support student engagement in scientific argumentation. Assessing the quality of scientific dialogic argumentation­ in these environments, however, has proven to be challenging. Existing analytic frameworks tend to assess scientific argumentation using the presence or absence of various types of comments (e.g., frequency of claims, rebuttals, and supporting statements) that do not fully convey the dynamic and dialogic nature of argumentation. In this chapter, we present a sequential analysis approach developed by Jeong (2005) that incorporates a coding scheme developed by Clark and Sampson (2007, 2008) to identify, visualize, and assess the dialogic processes of argumentation in online science learning environments in terms of transitional probabilities, transitional state diagrams, and other related measures. These measures include: (a) how and how often students respond to particular discourse moves (e.g., the probabilities that responses to claims are rebuttals vs. simple agreement vs. no response); and (b) how and to what extent observed response patterns produce extended chains of discourse moves that exhibit high levels of argumentation (claim → challenge → explain or amend claim). A sample analysis is presented to illustrate how this approach can also be used to assess how characteristics of the discourse environment affect the quality of argumentation and better understand the interplay between discourse environments and collaborative discourse.

New methods and software tools are needed to assess the quality of learners’ causal maps (maps that convey a learner’s understanding of complex phenomena) and the quality of learners’ discourse used to help justify changes and refinements in learners’ causal maps. New methods and software tools are needed to assess the dialog move sequences observed in group discourse that trigger changes in causal maps and to measure and visualize across time the extent to which changes in causal maps of the individual or collective group progress toward group consensus and target maps. The software tool called jMAP was developed to enable learners to individually produce and submit causal maps, download and aggregate the maps of other learners. It also generates aggregated maps to reveal similarities between individual/group maps, the percentage of maps sharing particular causal links, average causal strength assigned to each link, and degree of match between the maps of the collective group and the target/expert diagram. jMAP also supports the use of sequential analysis to measure and visualize (with transitional state diagrams) how learner’s causal maps change over time and how dialogic processes of argumentation conducted in online discussions trigger changes in learner’s causal maps. This chapter presents findings from two case studies to illustrate how jMAP can be used to support the assessment of causal understanding, and to identify areas for future research and development.

The first challenge of accomplishing the goals of any successful instructional system depends on accurately identifying characteristics of a particular learner or group of learners – such as the type and level of specific knowledge, skills, and other attributes. The second challenge is then leveraging the information to improve learning. This chapter is intended to extend current thinking about (a) educationally valuable skills and (b) instructional system design by describing an approach for analyzing key competencies and developing valid assessments embedded within an immersive game. Specifically, we will describe theoretically based research relating to stealth assessment, diagnosis, and instructional decisions, operational within an immersive game environment. Stealth assessment and diagnosis occur during the learning (playing) process, and instructional decisions are based on inferences of learners’ current and projected competency states. Inferences – both diagnostic and predictive – will be handled by Bayesian networks and used directly in student models to handle uncertainty via probabilistic inference to update and improve belief values on learner competencies. Resulting probabilities inform decision making, as needed in, for instance, the selection of instructional support based on the learner’s current state.

In this chapter, we describe our research and development efforts relating to eliciting, representing, and analyzing how individuals and small groups conceptualize complex problems. The methods described herein have all been developed and are in various states of being validated. In addition, the methods we describe have been automated and most have been integrated in an online model-based set of tools called HIMATT (Highly Interactive Model-based Assessment Tools and Technologies; available for research purposes at http://himatt.ezw.uni-freiburg.de/cgi-bin/hrun/himatt.pl and soon to be available on a server at Florida State University). HIMATT continues to expand in terms of the tools and technologies included. Our methods and tools represent an approach to learning and instruction that is now embedded in many of the graduate courses at Florida State University and also at the University of Freiburg. We call our approach model-based because it integrates representations of mental models and internal cognitive processes with tools that are used to (a) assess progress of learning, and (b) provide the basis for informative and reflective feedback during instruction.

This study tested the effects of linguistic qualifiers and intensifiers on the number and types of replies elicited per argument and per challenge posted in online debates. To facilitate collaborative argumentation, thirty-two students (22 females, 10 males) enrolled in a graduate-level online course classified and labeled their messages as arguments, challenges, supporting evidence, or explanations prior to posting each message. The findings showed that qualified arguments elicited 41 percent fewer replies (effect size = -.64), and the reduction in replies was greatest when qualified arguments were presented by females than males. Challenges without qualifiers, however, did not elicit more replies than challenges with qualifiers. These findings suggest that qualifiers were used to hedge arguments, and such behaviors should be discouraged during initial stages of identifying arguments (more so in all-female than in all-male groups) in order to elicit more diverse and more opposing viewpoints needed to thoroughly and critically analyze arguments. Keywords: Computer-mediated communication, CMC, communication style, group interaction patterns, interaction analysis, computer-supported collaborative learning, CSCL, collaborative argumentation.