Search

Purpose: This study aims to uncover divergent collaboration in makerspaces using social network analysis to examine ongoing social relations and sequential data pattern mining to invesitgate temporal changes in social activities. Design/methodology/approach: While there is a significant body of qualitative work on makerspaces, there is a lack of quantitative research identifying productive interactions in open-ended learning environments. This study explores the use of high frequency sensor data to capture divergent collaboration in a semester-long makerspace course, where students support each other while working on different projects. Findings: The main finding indicates that students who diversely mix with others performed better in a semester-long course. Additional results suggest that having a certain balance of working individually, collaborating with other students and interacting with instructors maximizes performance, provided that sufficient alone time is committed to develop individual technical skills. Research limitations/implications: These discoveries provide insight into how productive makerspace collaboration can occur within the framework of Divergent Collaboration Learning Mechanisms (Tissenbaum et al., 2017). Practical implications: Identifying the diversity and sequence of social interactions could also increase instructor awareness of struggling students and having this data in real-time opens new doors for identifying (un)productive behaviors. Originality/value: The contribution of this study is to explore the use of a sensor-based, data-driven, longitudinal approach in an ecologically valid setting to understand divergent collaboration in makerspaces. Finally, this study discusses how this work represents an initial step toward quantifying and supporting productive interactions in project-based learning environments.

The continuous advancement of natural user interfaces (NUIs) allows for the development of novel and creative ways to support collocated collaborative work in a wide range of areas, including teaching and learning. The use of NUIs, such as those based on interactive multi-touch surfaces and tangible user interfaces (TUIs), can offer unique opportunities for learning, for automatically capturing digital traces of students' multimodal interactions, and for facilitating multi-user exploration of student data. We used a composite framework to characterize our first-hand experiences and a small number of related deployments. The dimensions of analysis considered include the orchestration activities involved, the phases of pedagogical practice supported, the target actors, the iteration of the LA process, and the levels of impact of the LA deployment. Results from our analysis helped us identify the current trends, gaps, challenges, and pedagogical opportunities of the application of LA and the use of NUIs for supporting learning. [Paper presented at the LAK '16--International Conference on Learning Analytics and Knowledge (6th, Edinburgh, United Kingdom, April 25-29, 2016).]

Multimodal learning analytics (MMLA) research has made significant progress in modelling collaboration quality for the purpose of understanding collaboration behaviour and building automated collaboration estimation models. Deploying these automated models in authentic classroom scenarios, however, remains a challenge. This paper presents findings from an evaluation of collaboration quality estimation models. We collected audio, video and log data from two different Estonian schools. These data were used in different combinations to build collaboration estimation models and then assessed across different subjects, different types of activities (collaborative‐writing, group‐discussion) and different schools. Our results suggest that the automated collaboration model can generalize to the context of different schools but with a 25% degradation in balanced accuracy (from 82% to 57%). Moreover, the results also indicate that multimodality brings more performance improvement in the case of group‐discussion‐based activities than collaborative‐writing‐based activities. Further, our results suggest that the video data could be an alternative for understanding collaboration in authentic settings where higher‐quality audio data cannot be collected due to contextual factors. The findings have implications for building automated collaboration estimation systems to assist teachers with monitoring their collaborative classrooms. Practitioners notesWhat is already known about this topicMultimodal learning analytics researchers have established several features as potential indicators for collaboration quality, e.g., speaking time or joint visual attention.The current state of the art has shown the feasibility of building automated collaboration quality models.Recent research has provided preliminary evidence of the generalizability of developed automated models across contexts different in terms of given task and subject.What does this paper addThis paper offers collaboration indicators for different types of collaborative learning activities in authentic classroom settings.The paper includes a systematic investigation into collaboration quality automated model's generalizability across different tasks, types of tasks and schools.This paper also offers a comparison between different modalities' potential to estimate collaboration quality in authentic settings.Implications for practiceThe findings inform the development of automated collaboration monitoring systems for authentic classroom settings.This paper provides evidence on across‐school generalizability capabilities of collaboration quality estimation models. [ABSTRACT FROM AUTHOR]

Over the last decade, there has been a renewed interest in capturing twenty-first century skills using new data collection tools. In this article, we leverage an existing dataset where electrodermal activity (EDA) was used to identify markers of productive collaboration. The data came from 42 pairs of participants (N = 84) who had no coding experience and were asked to program a robot to solve a variety of mazes. Because little is known on how physiological synchrony relates to collaborative learning, we explored four different measures of synchrony: Signal Matching (SM), Instantaneous Derivative Matching (IDM), Directional Agreement (DA) and Pearson's Correlation (PC). Overall, we found PC to be positively associated with learning gains (r = 0.35) and DA with collaboration quality (r = 0.3). To gain further insights into these results, we also qualitatively analyzed two groups and identified situations with high or low physiological synchrony. We observed higher synchrony values when members of a productive group reacted to an external event (e.g., following instructions, receiving a hint), oscillations when they were watching a video or interacting with each other, and lower values when they were programming and / or seem to be confused. Based on these results, we developed a new measure of collaboration using electrodermal data: we computed the number of cycles between low and high synchronization. We found this measure to be significantly correlated with collaboration quality (r = 0.57) and learning gains (r = 0.47). This measure was not significantly correlated with the measures of physiological synchrony mentioned above, suggesting that it is capturing a different construct. We compare those results with prior studies and discuss implications for measuring collaborative process through physiological sensors.

In a previous study, we found that real-time mutual gaze perception (i.e., being able to see the gaze of your partner in real time on a computer screen while solving a learning task) had a positive effect on student collaboration and learning (Schneider & Pea, 2013). The goals of this paper are (1) to explore a variety of computational techniques for analyzing the transcripts of student discussions; (2) to examine whether any of those measures sheds new light on our previous results; and (3) to test whether those metrics have any predictive power regarding learning outcomes. Using various natural language processing algorithms, we found that linguistic coordination (i.e., the extent to which students mimic each other in terms of their grammatical structure) did not predict the quality of student collaboration or learning gains. However, we found that a simple computational measure of student verbal coherence (i.e., the extent to which students build on each other's ideas) was positively correlated with their learning gains. Additionally, this measure was significantly different across our experimental conditions: students who could see the gaze of their partner in real time were more likely to develop a coherent discussion. Finally, using various language metrics, we were able to roughly predict (i.e., using a median-split) learning gains with a 94.4% accuracy using Support Vector Machine. The accuracy dropped to 75% when we used our model on a validation set. We conclude by discussing the benefits of using computational techniques on educational datasets.

In this paper, we describe multimodal learning analytics (MMLA) techniques to analyze data collected around an interactive learning environment. In a previous study (Schneider & Blikstein, submitted), we designed and evaluated a Tangible User Interface (TUI) where dyads of students were asked to learn about the human hearing system by reconstructing it. In the current study, we present the analysis of the data collected in the form of logs, both from students' interaction with the tangible interface and as well as from their gestures, and we describe how we extracted meaningful predictors for student learning from these two datasets. First we show how Natural Language Processing (NLP) techniques can be used on the tangible interface logs to predict learning gains. Second, we explored how Kinect™ data can inform "in-situ" interactions around a tabletop by using clustering algorithms to find prototypical body positions. Finally, we fed those features to a machine-learning classifier (Support Vector Machine) and divided students in two groups after performing a median split on their learning scores. We found that we were able to predict students' learning gains (i.e. being above or belong the median split) with very high accuracy. We discuss the implications of these results for analyzing rich data from multimodal learning environments.

In this paper, we describe an experiment that compared the use of a Tangible User Interface (physical objects augmented with digital information) and a set of Contrasting Cases as a preparation for future learning. We carried out an experiment (N = 40) with a 2 × 2 design: the first factor compared traditional instruction ("Tell & Practice") with a constructivist activity designed using the Preparation for Future Learning framework (PFL). The second factor contrasted state-of-the-art PFL learning activity (i.e., students studying Contrasting Cases) with an interactive tabletop featuring digitally enhanced manipulatives. In agreement with prior work, we found that dyads of students who followed the PFL activity achieved significantly higher learning gains compared to their peers who followed a traditional "Tell & Practice" instruction (large effect size). A similar effect was found in favor of the interactive tabletop compared to the Contrasting Cases (small-to-moderate effect size). We discuss implications for designing socio-constructivist activities using new computer interfaces.

The Bifocal Modelling Framework (BMF) is an approach for science learning which links students' physical experimentation with computer modelling in real time, focusing on the comparison of the two media. In this paper, we explore how a Bifocal Modelling implementation supported learning outcomes related to both content and metamodeling knowledge, focusing on the role of designing models. Our study consisted of three conditions implemented with a total of 69 9th grade high-school students. The first and second classes were assigned two implementation modes of BMF: with and without a model design module. The third condition, employed as a control, consisted of a class that received instruction in the school's traditional approach. Our results indicate that students participating in both BMF implementations demonstrated improved content knowledge and a better understanding of metamodeling. However, only the 'BMF-with-design' group improved significantly in both content and metamodeling knowledge. Our qualitative analyses indicate that both BMF groups designed detailed models that included scientific explanations. However only students who engaged in the model design component: (1) completed a detailed model displaying molecular interaction; and (2) developed a critical perspective about models. We discuss the implications of those results for teaching scientific science concepts and metamodeling knowledge.

This commentary examines how the field of Multimodal Learning Analytics (MMLA; Blikstein, 2013) can support learning and teaching. More specifically, I focus on the challenges of implementing multimodal systems in real-world settings, and how lab findings can impact educational practices. I discuss the kinds of multimodal metrics we should pay attention to, how to capture them in ecological settings, and how the data can be shared with learners and teachers. This commentary proposes some first steps towards addressing these challenges and discusses related limitations and future work. [ABSTRACT FROM AUTHOR]

The goal of this paper is to bring new insights to the study of social learning processes by designing measures of collaboration using high-frequency sensor data. More specifically, we are interested in understanding the interplay between moments of collaboration and cooperation, which is an understudied area of research. We collected a multimodal dataset during a collaborative learning activity typical of makerspaces: learning how to program a robot. Pairs of participants were introduced to computational thinking concepts using a block-based environment. Mobile eye-trackers, physiological wristbands, and motion sensors captured their behavior and social interactions. In this paper, we analyze the eye-tracking data to capture participants' tendency to synchronize their visual attention. This paper provides three contributions: (1) we use an emerging methodology (mobile dual eye-tracking) to capture joint visual attention in a co-located setting and replicate findings that show how levels of joint visual attention are positively correlated with collaboration quality; (2) we qualitatively analyzed the co-occurrence of verbal activity and joint visual attention in low and high performing groups to better understand moments of collaboration and cooperation; (3) inspired by the qualitative observations and theories of collaborative learning, we designed a new quantitative measure that captures cycles of collaborative and cooperative work. Compared to simple measures of joint visual attention, we found it to increase correlation coefficients with learning and collaboration scores. We discuss those results and describe how advances in analyzing sensor data can contribute to theories of collaboration. We conclude with implications for capturing students' interactions in co-located spaces using Multimodal Learning Analytics (MMLA). [ABSTRACT FROM AUTHOR]

Technology is changing the way students interact with knowledge, and open-ended activities are one of the main types of tasks that students engage with in technology-rich environments. However, the amount of guidance needed to promote learning in these environments remains unknown. We explore this issue by focusing on the effects of step-by-step versus generic instructions on student's exploratory behavior and arousal levels. In this experiment, students completed three computer-based activities within a physics simulation software: building a tower, building a bridge and a free task. We did not find any effect of our experimental manipulation on students' task performance. We found, however, that detailed instruction induced higher level of activation followed by a relaxation phase and a recovery of the activation level in the last segment of the task (U-shaped curve). On the other hand, generic instructions seemed to lead students into a continuous relaxation pattern along the task (decreasing slope). Moreover, low and high-aroused students appear to be affected by the instructions differently, with high-aroused students at baseline showing less cognitive flexibility. Finally, we observed carryover effects, where types of instruction kept influencing students' levels of activation in a following open-ended task. We discuss implications of those results for designing learning activities in constructionist, technology-rich environments.

We describe preliminary applications of network analysis techniques to eye-tracking data collected during a collaborative learning activity. This paper makes three contributions: first, we visualize collaborative eye-tracking data as networks, where the nodes of the graph represent fixations and edges represent saccades. We found that those representations can serve as starting points for formulating research questions and hypotheses about collaborative processes. Second, network metrics can be computed to interpret the properties of the graph and find proxies for the quality of students' collaboration. We found that different characteristics of our graphs correlated with different aspects of students' collaboration (for instance, the extent to which students reached consensus was associated with the average size of the "strongly connected components" of the graphs). Third, we used those characteristics to predict the quality of students' collaboration by feeding those features into a machine-learning algorithm. We found that among the eight dimensions of collaboration that we considered, we were able to roughly predict (using a median-split) students' quality of collaboration with an accuracy between ~85 and 100%. We conclude by discussing implications for developing "collaboration-sensing" tools, and comment on implementing this approach for formal learning environments.

In this paper we present the results of an eye-tracking study on collaborative problem-solving dyads. Dyads remotely collaborated to learn from contrasting cases involving basic concepts about how the human brain processes visual information. In one condition, dyads saw the eye gazes of their partner on the screen; in a control group, they did not have access to this information. Results indicated that this real-time mutual gaze perception intervention helped students achieve a higher quality of collaboration and a higher learning gain. Implications for supporting group collaboration are discussed.