Search

Computer programs correspond to instructions that tell a computer how to perform a task. Because computers do not understand human languages, programs have to be written in a specialized programming language. Such languages have unique syntax and semantics that on the surface appear quite different from human languages. To foster understanding what a program means, novices need to translate the programming language instructions into a familiar human language. We developed a tutoring system that provides translation activities, from the language Python into plain English, and feedback on student answers. One version of the tutoring system provides open-ended input boxes that students type translations into. A second version instead asks students to generate answers to the translation using drop-down menus. The latter provides more scaffolding but also may encourage shallow behaviors such as guessing and checking. To shed light on how these designs influence learning, we evaluate them with novice programmers. Here, we report on the results of the evaluation and next steps.

A fundamental skill needed to learn to program is code tracing, which involves simulating the steps a computer takes when executing a program to predict how it will behave and what output it. While code tracing helps students understand and write programs, students find this activity challenging. However, to date, little work exists on how to best support novice programmers in this activity. To address this gap, the high-level goal of our research is to investigates the utility of peer-tutoring for learning to trace programs. In our prior work, we conducted a preliminary case study exploring qualitative features of peer tutoring interactions. As the next step, we conducted an experimental study with two conditions: code tracing in a peer tutoring context vs. code-tracing alone. We present results from this study, focusing on quantitative results related to learning outcomes in each condition.

A notional machine is an abstract representation of how a computer works. Prior research shows that students often form incomplete or inaccurate mental models of the notional machine, reflecting misunderstandings and causing programming errors. To date, however, there is very little work on how to help students acquire accurate models of the notional machine. To fill this gap, this research focuses on novice programmers and experimentally investigates the impact of direct instruction of the notional machine compared to standard instruction that does not include the notional machine. The instruction is embedded in an online computer tutor that we implemented and compared to a version of the tutor that does not refer to the notional machine but otherwise includes the same materials. In particular, both tutor versions include instructional videos followed by self-explanation prompts. We will present data from an evaluation of both tutor versions.

Programming is a complex problem-solving domain, requiring the coordination of different types of knowledge and skills. The present study investigates expert and novice programming problem solving by analyzing talk-aloud transcripts and the code generated. Based on this analysis a set of basic goal and step components used by novice and expert programmers are identified, which will inform on the generation of cognitive models in the next phase of this research.

We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

A mental model is an internal representation that explains how something works. Mental model construction is facilitated by self-explanation, the active generation of explanations for oneself. The overarching goal of this research is to empirically investigate the utility of self-explanation for developing mental models when learning to program. Programming is notoriously challenging and, despite evidence of the importance of mental models for learning, little work has focused on mental models of students learning how to program. They need correct mental models of the notional machine, an abstraction of the steps taken by a computer as it processes a program. Because students do not spontaneously self-explain, we are using a user-centered approach to design a computer tutor to prompt for self-explanation about the notional machine. Here, we present qualitative results on students’ interactions with an initial version of the tutor, including the form of their self-explanations and corresponding mental models.

Productive failure (PF) is a learning paradigm that reverses the standard order of instruction by asking students to solve problems prior to instruction. This paradigm has been shown to be effective for fostering student learning. To date, however, the role of student emotion in productive failure has not been investigated. In other paradigms, there is some evidence that failure elicits negative emotions and that these emotions can interfere with learning. This leads to a conundrum given productive failure’s positive effect on learning. To shed light on this, we report on results from a study (N = 48) in the productive failure paradigm. For the analysis, we used a mixed-methods approach to investigate the distribution of emotions in productive failure, how these changed across different instructional activities, and the relation between emotions and posttest performance.

The goal of the present study was to use content analysis to gain insight into the process of problem-solving of novice and expert programmers. While classic work on programmers identifies goals / plans as key constructs needed to code, there is relatively little work using protocol analysis. We recruited 7 expert and 12 novice rogrammers who completed up to 3 brief programming problems while providing a talk-aloud of their inner problem solving process. Based on analysis of the transcriptions of this talk aloud data, we identified the goals and steps used, as well as the broad differences between experts and novices in their problem solving process. These differences were formalized into python ACT-R models, and model output was compared to programs written by human participants.

Learning to program is challenging, because it involves novel skills. In contrast to the majority of work focusing on code generation, we target the skill of code tracing. Code tracing involves simulating the high-level actions a computer takes when it executes the program, including the flow of execution through it and how variable values change as a result. Code tracing supports program comprehension and generation. However, many students do not code trace effectively. Our project investigates the utility of peer tutoring for the learning of code tracing. While this approach remains untested in this context, it has successfully been used in other domains to improve student learning. We will present qualitative data from a case study involving students tutoring each other to code trace. This work is the first step in the context of a broader project focused on identifying ways to help students learn to code.

In today's world, IT security plays a critical role in different organizations, yet little is known about IT security in the context of organizations. This paper addresses this issue based on qualitative description analysis of 10 interviews with IT security practitioners from small to medium size organizations. Our results revealed the required knowledge and skills for effective IT security, IT security tasks, and the tools which are used to perform these tasks. Based on these results, we realized that current IT security tools can be improved in order to provide more effective support for security practitioners' activities. Founded on our analysis, we proposed some guidelines, such as regular update, settings for alerts, integrity of data, and effective logging of forensics information, in order to improve IT security tools.

This study investigates how security practitioners perform diagnostic work during the identification of security incidents. Based on empirical data from 16 interviews with security practitioners, we identify the tasks, skills, strategies and tools that security practitioners use to diagnose security incidents. Our analysis shows that diagnosis is a highly collaborative activity, which may involve practitioners developing their own tools to perform specific tasks. Our results also show that diagnosis during incident response is complicated by practitioners' need to rely on tacit knowledge, as well as usability issues with security tools. We offer recommendations to improve technology that supports the diagnosis of security incidents.

Given that Information Technology (IT) has become pervasive in today's organizations, properly securing systems is critical. However, many challenges remain with respect to implementing sound technologies and security processes. In the past decade, the research focus has shifted, from only considering pure technological factors, to also include organizational and human factors, as these latter two factors play a key role in influencing security practices and outcomes. The corresponding research area, namely HCISec, explores how to design "usable security" solutions for all users. To date, however, little work has targeted the population that is at the crossroads of the above mentioned human/organizational/technological factors, namely security professionals, who are responsible for protecting their organizations from IT-related threats. Currently, these individuals lack sufficient support, as is evident by the rising number and cost of incidents. As a first step in gaining insight on how to provide this support, we are conducting a field study that involves gathering data from security administrators working in a variety of organizations. Here, we will present findings from our study, which are based on inductive qualitative analysis, and include descriptive models of the unique needs of this highly specialized population, the challenges it faces, and the sub-optimal situations that lead to security incidents. Based on our findings, we propose some guidelines for designing effective "usable security" solutions.

The usability of security systems within an organization is impacted not only by tool interfaces but also by the security management model (SMM) of the IT security team. Finding the right SMM is critical and yet can be challenging, as there are tradeoffs inherent with each approach. We present a case study of one post-secondary educational institution that created a centralized security team, but disbanded it in favour of a more distributed approach three years later. The case study consists of interviews with ten IT staff from across the organization who gave us their diverse perspectives of the realities of managing security in a decentralized post-secondary organization. We contrast this organization's experiences with SMMS with expectations from industry standards and derive organizational factors that impact the success of the models. These factors highlight the importance of considering both the organization's security goals as well as its structure when evaluating potential SMMs. Furthermore, top management support, security policies, and a security team with vested authority, along with the organization's prior security management history, impact the success of a given SMM.

We report factors differentiating security and other IT responsibilities. Our findings are based on a qualitative analysis of data from 27 interviews across 11 distinct organizations. The results show that compared to general IT, security professionals have to manage a higher level of complexity, stemming from factors such as the need to balance usability and security, negative stakeholder perception, and external threats. We synthesize the differences into an overall model and discuss how the model can guide the development of support for professionals with security responsibilities.