Search

In the past, team coordination dynamics have been explored using nonlinear dynamical systems (NDS) methods, but the relationship between team coordination dynamics and team performance for all-human teams was assumed to be linear. The current study examines team coordination dynamics with an extended version of the NDS methods and assumes that its relationship with team performance for human-autonomy teams (HAT) is nonlinear. In this study, three team conditions are compared with the goals of better understanding how team coordination dynamics differ between all-human teams and HAT and how these dynamics relate to team performance and team situation awareness. Each condition was determined based on manipulation of the pilot role: in the first condition (synthetic) the pilot role was played by a synthetic agent, in the second condition (control) it was a randomly assigned participant, and in the third condition (experimenter) it was an expert who used a role specific coordination script. NDS indices revealed that synthetic teams were rigid, followed by experimenter teams, who were metastable, and control teams, who were unstable. Experimenter teams demonstrated better team effectiveness (i.e., better team performance and team situation awareness) than control and synthetic teams. Team coordination stability is related to team performance and team situation awareness in a nonlinear manner with optimal performance and situation awareness associated with metastability coupled with flexibility. This result means that future development of synthetic teams should address these coordination dynamics, specifically, rigidity in coordination.

The assessment of argumentative writing generally includes analyses of the specific linguistic and rhetorical features contained in the individual essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing may more accurately capture their proficiency. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six argumentative essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the supposition that skilled writers demonstrate linguistic flexibility across the argumentative essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in argumentative writing skill and develops a strong foundation on which to conduct future research and educational interventions.

This study investigated the effect of incorporating spelling and grammar checking tools within an automated writing tutoring system, Writing Pal. High school students (n = 119) wrote and revised six persuasive essays. After initial drafts, all students received formative feedback about writing strategies. Half of the participants were also given access to spelling and grammar checking tools during the writing and revision periods. Linear mixed effects models revealed that essay quality for students in both conditions improved from initial draft to revision in terms of all aspects except essay unity. The availability of spelling and grammar checking yielded added improvements from initial draft to revision for several aspects of essay quality (i.e., conclusion, organization, voice, grammar/mechanics, and word choice), but other aspects were unaffected (i.e., introduction, body, unity, and sentence structure). The availability of spelling and grammar checking tools had no effect on holistic essay scores. These results indicate that automated spelling and grammar feedback contribute to modest, incremental improvements in writing quality that may complement automated strategy feedback.

Game-based environments frequently afford students the opportunity to exert agency over their learning paths by making various choices within the environment. The combination of log data from these systems and dynamic methodologies may serve as a stealth means to assess how students behave (i.e., deterministic or random) within these learning environments. The current work captures variations in students’ behavior patterns by employing two dynamic analyses to classify students’ sequences of choices within an adaptive learning environment. Random Walk analyses and Hurst exponents were used to classify students’ interaction patterns as random or deterministic. Forty high school students interacted with the game-based system, iSTART-ME, for 11-sessions (pretest, 8 training sessions, posttest, and a delayed retention test). Analyses revealed that students who interacted in a more deterministic manner also generated higher quality self-explanations during training sessions. The results point toward the potential for dynamic analyses such as random walk analyses and Hurst exponents to provide stealth assessments of students’ learning behaviors while engaged within a game-based environment.

Self-explanations are commonly used to assess on-line reading comprehension processes. However, traditional methods of analysis ignore important temporal variations in these explanations. This study investigated how dynamical systems theory could be used to reveal linguistic patterns that are predictive of self-explanation quality. High school students (n = 232) generated self-explanations while they read a science text. Recurrence Plots were generated to show qualitative differences in students' linguistic sequences that were later quantified by indices derived by Recurrence Quantification Analysis (RQA). To predict self-explanation quality, RQA indices, along with summative measures (i.e., number of words, mean word length, and type-token ration) and general reading ability, served as predictors in a series of regression models. Regression analyses indicated that recurrence in students' self-explanations significantly predicted human rated self-explanation quality, even after controlling for summative measures of self-explanations, individual differences, and the text that was read (R2 = 0.68). These results demonstrate the utility of RQA in exposing and quantifying temporal structure in student's self-explanations. Further, they imply that dynamical systems methodology can be used to uncover important processes that occur during comprehension.

The assessment of writing proficiency generally includes analyses of the specific linguistic and rhetorical features contained in the singular essays produced by students. However, researchers have recently proposed that an individual's ability to flexibly adapt the linguistic properties of their writing might more closely capture writing skill. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the fact that developing writers demonstrate linguistic flexibility across the essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students' essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in writing skill and develops a strong foundation on which to conduct future research and educational interventions.

In this study, we investigated the degree to which the cognitive processes in which students engage during reading comprehension could be examined through dynamical analyses of their natural language responses to texts. High school students (n = 142) generated typed self-explanations while reading a science text. They then completed a comprehension test that measured their comprehension at both surface and deep levels. The recurrent patterns of the words in students' self-explanations were first visualized in recurrence plots. These visualizations allowed us to qualitatively analyze the different self-explanation processes of skilled and less skilled readers. These recurrence plots then allowed us to calculate recurrence indices, which represented the properties of these temporal word patterns. Results of correlation and regression analyses revealed that these recurrence indices were significantly related to the students' comprehension scores at both surface- and deep levels. Additionally, when combined with summative metrics of word use, these indices were able to account for 32% of the variance in students' overall text comprehension scores. Overall, our results suggest that recurrence quantification analysis can be utilized to guide both qualitative and quantitative assessments of students' comprehension.

Teamwork is a complex dynamic process that emerges from team member interaction. The dynamics provide a characterization of the team over time. The stability, flexibility, and resilience of team dynamics over various windows of time can change with experience, training, environmental perturbations, and technological intervention. Once patterns of team dynamics have been established for a particular team, anomalous dynamics can signify impending teamwork problems. Dynamical systems modeling has been applied to many physical and natural systems and has only recently been applied to teamwork. The panel that we have assembled has applied dynamical modeling to this problem from different perspectives and at different scales. Each panelist will overview his or her approach to this problem. We will then discuss pros and cons of each approach and possibilities for using them in a complementary fashion.

Project overview. The current study focuses on the nature of team coordination dynamics within all-human teams and Human-Autonomy Teams (HAT) in the context of the development of a fully-fledged synthetic agent that is a computational cognitive model for a three-agent Unmanned Aircraft System (UAS) ground crew. In this study, the relationship between team coordination dynamics and team performance within the HAT and all-human teams is considered. To serve as a teammate, the synthetic agent must be able to communicate and coordinate with its human teammates in a constructive and timely manner (Demir, McNeese, & Cooke, 2016). In this current research, there were three heterogeneous team members who communicated via a text-based communication system to photograph target waypoints. Each team member had a different role: (1) navigator – provides information regarding a flight plan with speed and altitude restrictions of each waypoint; (2) pilot – controls the UAS by adjusting its altitude and airspeed by negotiating with the photographer to take a good photo for the target waypoints; and (3) photographer – screens camera settings, and sends feedback to the other team members regarding the status of target’s photograph. At each target waypoint, this coordination sequence among the team members, called Information-Negotiation-Feedback (INF), is captured by a Kappa Score (Gorman, Amazeen, & Cooke, 2010) that describes the sequence and timing of the information coordination. Three conditions were created that manipulated the pilot role: (1) Synthetic – pilot was the synthetic agent, (2) Control – pilot was a randomly assigned participant, and (3) Experimenter – pilot was an experimenter who was highly experienced with the task and focused on pushing and pulling the information in a timely manner. Method. In this experiment, there were 30 teams (ten teams for each condition): control teams were composed of three participants randomly assigned to each role; synthetic and experimenter teams were composed of only two participants randomly assigned to the navigator and photographer roles. The experiment consisted of five missions (each 40 minutes) in which teams needed to take as many “good” photos as possible of ground targets while avoiding alarms and rule violations. Several measures were obtained from this research, including team performance scores (mission and target level), team process measures (situation awareness, process ratings, communication and coordination), and other measures (teamwork knowledge, workload, and demographics). The research reported here identifies how differences in team coordination, captured by Kappa, relate to performance of all human teams and HAT teams. In this paper, we focus on: (1) target level team performance scores calculated based on the time spent inside a target waypoint to get a good photo; and (2) two team coordination dynamics measures: stability and team communication determinism. Stability was inversely related to the largest Lyapunov Exponent which was estimated by Kappa, that is, the INF coordination sequence. Team communication determinism was estimated from communication data using Joint Recurrence Quantification Analysis (Marwan, Carmen, Thiel, & Kurths, 2007) and served as an index of flexible behavior. Results and discussion. In general, findings indicate that (1) synthetic teams were most stable, followed by experimenter teams, who were moderately stable, and control teams, who were least stable; and (2) extreme stability and instability corresponded to lower levels of performance; experimenter teams performed best, followed by control teams and, then synthetic teams. Thus, synthetic agents could be made more effective if interventions were developed to enhance the flexibility and adaptive nature of HATs (Demir, 2017).