Search

Previous research has revealed that newcomer women are disproportionately affected by gender-biased barriers in open source software (OSS) projects. However, this research has focused mainly on social/cultural factors, neglecting the software tools and infrastructure. To shed light on how OSS tools and infrastructure might factor into OSS barriers to entry, we conducted two studies: (1) a field study with five teams of software professionals, who worked through five use cases to analyze the tools and infrastructure used in their OSS projects; and (2) a diary study with 22 newcomers (9 women and 13 men) to investigate whether the barriers matched the ones identified by the software professionals. The field study produced a bleak result: software professionals found gender biases in 73% of all the newcomer barriers they identified. Further, the diary study confirmed these results: Women newcomers encountered gender biases in 63% of barriers they faced. Fortunately, many the kinds of barriers and biases revealed in these studies could potentially be ameliorated through changes to the OSS software environments and tool.

Version Control Systems (VCS) are an important source of information for developers. This calls for a principled understanding of developers’ information seeking in VCS—both for improving existing tools and for understanding requirements for new tools. Our prior work investigated empirically how and why developers seek information in VCS: in this paper, we complement and enrich our prior findings by reanalyzing the data via a theory's lens. Using the lens of Information Foraging Theory (IFT), we present new insights not revealed by the prior empirical work. First, while looking for specific information, participants' foraging behaviors were consistent with other foraging situations in SE; therefore, prior research on IFT-based SE tool design can be leveraged for VCS. Second, in change awareness foraging, participants consumed similar diets, but in subtly different ways than in other situations; this calls for further investigations into change awareness foraging. Third, while committing changes, participants attempted to enable future foragers, but the competing needs of different foraging situations led to tensions that participants failed to balance: this opens up a new avenue for research at the intersection of IFT and SE, namely, creating forageable information. Finally, the results of using an IFT lens on these data provides some evidence as to IFT's scoping and utility for the version control domain.

Assessing and understanding intelligent agents is a difficult task for users who lack an AI background. “Explainable AI” (XAI) aims to address this problem, but what should be in an explanation? One route toward answering this question is to turn to theories of how humans try to obtain information they seek. Information Foraging Theory (IFT) is one such theory. In this article, we present a series of studies 1 using IFT: the first investigates how expert explainers supply explanations in the RTS domain, the second investigates what explanations domain experts demand from agents in the RTS domain, and the last focuses on how both populations try to explain a state-of-the-art AI. Our results show that RTS environments like StarCraft offer so many options that change so rapidly, foraging tends to be very costly. Ways foragers attempted to manage such costs included “satisficing” approaches to reduce their cognitive load, such as focusing more on What information than on Why information, strategic use of language to communicate a lot of nuanced information in a few words, and optimizing their environment when possible to make their most valuable information patches readily available. Further, when a real AI entered the picture, even very experienced domain experts had difficulty understanding and judging some of the AI’s unconventional behaviors. Finally, our results reveal ways Information Foraging Theory can inform future XAI interactive explanation environments, and also how XAI can inform IFT.

Gender inclusivity in software is gaining attention from researchers and practitioners, with some seeing it as a nonfunctional requirement. To investigate how gender inclusivity can be incorporated into creating software, we gathered data during periods ranging from 5 months to 3.5 years from 10 software teams that used the Gender Inclusiveness Magnifier (GenderMag) to achieve the gender inclusivity quality attribute. GenderMag is a method for detecting and fixing gender inclusivity issues in software. In this article, we summarize several practices the teams devised and pitfalls they encountered.

“In what circumstances would you want this AI to make decisions on your behalf?” We have been investigating how to enable a user of an Artificial Intelligence‐powered system to answer questions like this through a series of empirical studies, a group of which we summarize here. We began the series by (a) comparing four explanation configurations of saliency explanations and/or reward explanations. From this study we learned that, although some configurations had significant strengths, no one configuration was a clear “winner.” This result led us to hypothesize that one reason for the low success rates Explainable AI (XAI) research has in enabling users to create a coherent mental model is that the AI itself does not have a coherent model. This hypothesis led us to (b) build a model‐based agent, to compare explaining it with explaining a model‐free agent. Our results were encouraging, but we then realized that participants' cognitive energy was being sapped by having to create not only a mental model, but also a process by which to create that mental model. This realization led us to (c) create such a process (which we term After‐Action Review for AI or “AAR/AI”) for them, integrate it into the explanation environment, and compare participants' success with AAR/AI scaffolding vs without it. Our AAR/AI studies' results showed that AAR/AI participants were more effective assessing the AI than non‐AAR/AI participants, with significantly better precision and significantly better recall at finding the AI's reasoning flaws.

Imbuing furniture with robot properties reduces the physical labor and time needed for arranging spaces, such as homes, classrooms, and offices. Outsourcing labor tasks to robot furniture requires users’ involvement with functional user interfaces. We performed a user study on multi-robot furniture and added additional features based on the study results. The study involved 12 participants rearranging multiple non-robotic and robotic chairs (ChairBots). Results from the video and interview analysis revealed five high-level features missing in the original ChairBot: dual screen-based user interface, the ability to save and to set arrangements, the ability to move in multi-robot formations, the ability to snap to angles/gridlines, and higher movement precision. The improved system allows users to control multiple furniture robots, both locally and remotely. Such improvement sets the baseline functionalities of robot furniture arrangement systems while extending the potential utilization of established robotic chairs.

Web-active end-user programmers squander much of their time foraging for bugs and related information in mashup programming environments as well as on the web. To analyze this foraging behavior while debugging, we utilize an Information Foraging Theory perspective. Information Foraging Theory models the human (predator) behavior to forage for specific information (prey) in the webpages or programming IDEs (patches) by following the information features (cues) in the environment. We qualitatively studied the debugging behavior of 16 web-active end users. Our results show that end-user programmers spend substantial amounts (73%) of their time just foraging. Further, our study reveals new cue types and foraging strategies framed in terms of Information Foraging Theory, and it uncovers which of these helped end-user programmers succeed in their debugging efforts.

The tools and infrastructure used in tech, including Open Source Software (OSS), can embed "inclusivity bugs"---features that disproportionately disadvantage particular groups of contributors. To see whether OSS developers have existing practices to ward off such bugs, we surveyed 266 OSS developers. Our results show that a majority (77%) of developers do not use any inclusivity practices, and 92% of respondents cited a lack of concrete resources to enable them to do so. To help fill this gap, this paper introduces AID, a tool that automates the GenderMag method to systematically find gender-inclusivity bugs in software. We then present the results of the tool's evaluation on 20 GitHub projects. The tool achieved precision of 0.69, recall of 0.92, an F-measure of 0.79 and even captured some inclusivity bugs that human GenderMag teams missed.

How should empirical researchers conduct controlled, remote “lab” studies in the uncontrolled, noisy conditions of each participant's own home? Volatility in participant home environments, hardware, internet connection, and surrounding distractions takes the “controlled” out of controlled studies. This paper recounts our in-the-trenches mitigations for designing and conducting two complex controlled studies under COVID, in which participants, from home, interactively localized faults in an AI system. The studies with our COVID-era mitigations in 5 categories—Privacy/Security, Data Collection, Control, Technology Issues, Payment-ultimately produced crisp results beyond what we thought possible under such uncontrolled circumstances.

Artificial Intelligence (AI) and End-User Development (EUD) look at automation from two different perspectives. The former tends to provide fully automatic solutions, the latter aims to empower users to directly create what they want. We need both, but it is still unclear how to combine them to obtain effective every-day automations that meet the flexible and dynamic user needs. The panel aims to stimulate the Human-Computer Interaction community to think more carefully about such aspects and the possible approaches to address them.

How can user interface and user experience (UI/UX) professionals assess whether their software supports diverse users? And if they find problems, how can they fix them? We begin this keynote address with a summary of GenderMag, a systematic inspection method for finding and fixing "gender inclusivity bugs---biases against different genders in software interfaces and workflows. We then show what UI/UX professionals are doing with it in the real world, from their bias finds & fixes to their practices & pitfalls in using it. Finally, we present InclusiveMag, a meta-method that can be used by HCI researchers to generate systematic inclusiveness methods for other dimensions of diversity.

Although the need for gender-inclusivity in software is gaining attention among SE researchers and SE practitioners, and at least one method (GenderMag) has been published to help, little has been reported on how to make such methods work in real-world settings. Real-world teams are ever-mindful of the practicalities of adding new methods on top of their existing processes. For example, how can they keep the time costs viable? How can they maximize impacts of using it? What about controversies that can arise in talking about gender? To find out how software teams "in the trenches" handle these and similar questions, we collected the GenderMag-based processes of 10 real-world software teams---more than 50 people---for periods ranging from 5 months to 3.5 years. We present these teams' insights and experiences in the form of 9 practices, 2 potential pitfalls, and 2 open issues, so as to provide their insights to other real-world software teams trying to engineer gender-inclusivity into their software products.

Explainable AI (XAI) is growing in importance as AI pervades modern society, but few have studied how XAI can directly support people trying to assess an AI agent. Without a rigorous process, people may approach assessment in ad hoc ways---leading to the possibility of wide variations in assessment of the same agent due only to variations in their processes. AAR, or After-Action Review, is a method some military organizations use to assess human agents, and it has been validated in many domains. Drawing upon this strategy, we derived an AAR for AI, to organize ways people assess reinforcement learning (RL) agents in a sequential decision-making environment. The results of our qualitative study revealed several strengths and weaknesses of the AAR/AI process and the explanations embedded within it.

Many systems are designed to help novices who want to learn programming, but few support those who are not necessarily interested in learning programming. This paper targets the subset of end-user programmers (EUPs) in this category. We present a set of principles on how to help EUPs like this learn just a little when they need to overcome a barrier. We then instantiate the principles in a prototype and empirically investigate them in three studies: a formative think-aloud study, a pair of summer camps attended by 42 teens, and a third summer camp study featuring a different environment attended by 48 teens. Finally, we present a generalized architecture to facilitate the inclusion of Idea Gardens into other systems, illustrating with examples from Idea Garden prototypes. Results have been very encouraging. For example, under our principles, Study #2s camp participants required significantly less in-person help than in a previous camp to learn the same amount of material in the same amount of time. The Idea Garden, based on 7 principles, supports several dimensions of diversity.The Idea Garden helps stuck EUPs by providing just-in-time problem-solving support.Three separate environments have hosted versions of the Idea Garden.Each version was empirically evaluated for effectiveness.The Idea Garden can be ported to other environments via a generalized architecture.

How can software practitioners assess whether their software supports diverse users? Although there are empirical processes that can be used to find "inclusivity bugs" piecemeal, what is often needed is a systematic inspection method to assess soft-ware's support for diverse populations. To help fill this gap, this paper introduces InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for a particular dimension of diversity. We then present a multi-case study covering eight diversity dimensions, of eight teams' experiences applying InclusiveMag to eight under-served populations and their "mainstream" counterparts.

Refereed/Peer-reviewed

Building software systems is hard work, with challenges ranging from technical issues to usability issues. If the technical issues are not addressed, the software cannot work - but if the usability issues are not addressed, many potential users and customers are not even interested in whether it works. Further, usability must be inclusive: software needs to support diverse sorts of users. To help software professionals address gender-inclusive usability, we have created the GenderMag Recorder's Assistant tool. This Open Source tool is the first to semi-automate evaluating gender biases in software that is being designed, developed, or maintained. In this showpiece, we will demo the tool and encourage attendees to get involved in using it and improving upon it.

How can we support software professionals who want to build human-adaptive sociotechnical systems? Building such systems requires skills some developers may lack, such as applying human-centric concepts to the software they develop and/or mentally modeling other people. Effective socio-technical methods exist to help, but most are manual and cognitively burdensome. In this paper, we investigate ways semi-automating a socio-technical method might help, using as our lens GenderMag, a method that requires people to mentally model people with genders different from their own. Toward this end, we created the GenderMag Recorder's Assistant, a semi-automated visual tool, and conducted a small field study and a 92-participant controlled study. Results of our investigation revealed ways the tool helped with cognitive load and ways it did not; unforeseen advantages of the tool in increasing participants' engagement with the method; and a few unforeseen advantages of the manual approach as well.

This position paper considers what studying Open Source Software tools can lend to understanding the topic of Gender Diversity in Open Source Software. More specifically we investigate the GenderMag method, a Gender Inclusive method and how it can help increase gender inclusiveness in the tools that are used by OSS communities.

Assessing and understanding intelligent agents is a difficult task for users that lack an AI background. A relatively new area, called "Explainable AI," is emerging to help address this problem, but little is known about how users would forage through information an explanation system might offer. To inform the development of Explainable AI systems, we conducted a formative study, using the lens of Information Foraging Theory, into how experienced users foraged in the domain of StarCraft to assess an agent. Our results showed that participants faced difficult foraging problems. These foraging problems caused participants to entirely miss events that were important to them, reluctantly choose to ignore actions they did not want to ignore, and bear high cognitive, navigation, and information costs to access the information they needed., 13 pages, 10 figures, to appear in ACM IUI 2018

Research has emerged over the past decade showing gender biases in software. Although a few methods and prototype systems have emerged to help address this issue, none have been reported to have an impact on the people who actually build software. In this paper, we summarize a few highlights from a year-long field study investigating how Gender HCI methods to address gender biases in software can make impacts on a large software company.

Although Information Foraging Theory (IFT) research for desktop environments has provided important insights into numerous information foraging tasks, we have been unable to locate IFT research for mobile environments. Despite the limits of mobile platforms, mobile apps are increasingly serving functions that were once exclusively the territory of desktops — and as the complexity of mobile apps increases, so does the need for foraging. In this paper we investigate, through a theory-based, dual replication study, whether and how foraging results from a desktop IDE generalize to a functionally similar mobile IDE. Our results show ways prior foraging research results from desktop IDEs generalize to mobile IDEs and ways they do not, and point to challenging open research questions for foraging on mobile environments.

Foraging among similar variants of the same artifact is a common activity, but computational models of Information Foraging Theory (IFT) have not been developed to take such variants into account. Without being able to computationally predict people's foraging behavior with variants, our ability to harness the theory in practical ways--such as building and systematically assessing tools for people who forage different variants of an artifact--is limited. Therefore, in this paper, we introduce a new predictive model, PFIS-V, that builds upon PFIS3, the most recent of the PFIS family of modeling IFT in programming situations. Our empirical results show that PFIS-V is up to 25% more accurate than PFIS3 in predicting where a forager will navigate in a variationed information space.

How should an AI-based explanation system explain an agent's complex behavior to ordinary end users who have no background in AI? Answering this question is an active research area, for if an AI-based explanation system could effectively explain intelligent agents' behavior, it could enable the end users to understand, assess, and appropriately trust (or distrust) the agents attempting to help them. To provide insights into this question, we turned to human expert explainers in the real-time strategy domain, "shoutcaster", to understand (1) how they foraged in an evolving strategy game in real time, (2) how they assessed the players' behaviors, and (3) how they constructed pertinent and timely explanations out of their insights and delivered them to their audience. The results provided insights into shoutcasters' foraging strategies for gleaning information necessary to assess and explain the players; a characterization of the types of implicit questions shoutcasters answered; and implications for creating explanations by using the patterns, Comment: 12 pages, 11 figures, submitted to CHI 2017

One area of research in the end-user development area is known as end-user software engineering (EUSE). Research in EUSE aims to invent new kinds of technologies that collaborate with end users to improve the quality of their software. EUSE has become an active research area since its birth in the early 2000s, with a large body of literature upon which EUSE researchers can build. However, building upon these works can be difficult when projects lack connections due to an absence of cross-cutting foundations to tie them together. In this chapter, we advocate for stronger theory foundations and show the advantages through three theory-oriented projects: (1) the Explanatory Debugging approach, to help end users debug their intelligent assistants; (2) the GenderMag method, which identifies problems with gender inclusiveness in EUSE tools and other software; and (3) the Idea Garden approach, to help end users to help themselves in overcoming programming barriers. In each of these examples, we show how having a theoretical foundation facilitated generalizing beyond individual tools to the production of general methods and principles for other researchers to directly draw upon in their own works.

Although there have been many advances in end-user programming environments, recent empirical studies report that programming still remains difficult for end users. We hypothesize that one reason may be lack of effective support for helping end-user programmers problem-solve their own way around barriers they encounter. Therefore, in this paper, we describe the Idea Garden, a concept designed to help end-user programmers generate new ideas and problem-solve when they run into barriers. The Idea Garden has its roots in Minimalist Learning Theory and problem-solving theories. Our proof-of-concept prototype of the Idea Garden concept in the CoScripter end-user programming environment currently targets three barriers reported in end-user programming literature. It does so using an integrated, just-in-time combination of scaffolding for problem-solving strategies, for design patterns, and for programming concepts. Our empirical results showed that this approach helped enduser programmers overcome all three types of barriers our prototype targeted.

Theories of human behavior are an important but largely untapped resource for software engineering research. They facilitate understanding of human developers’ needs and activities, and thus can serve as a valuable resource to researchers designing software engineering tools. Furthermore, theories abstract beyond specific methods and tools to fundamental principles that can be applied to new situations. Toward filling this gap, we investigate the applicability and utility of Information Foraging Theory (IFT) for understanding information-intensive software engineering tasks, drawing upon literature in three areas: debugging, refactoring, and reuse. In particular, we focus on software engineering tools that aim to support information-intensive activities, that is, activities in which developers spend time seeking information. Regarding applicability, we consider whether and how the mathematical equations within IFT can be used to explain why certain existing tools have proven empirically successful at helping software engineers. Regarding utility, we applied an IFT perspective to identify recurring design patterns in these successful tools, and consider what opportunities for future research are revealed by our IFT perspective.

Many theories of human debugging rely on complex mental constructs that offer little practical advice to builders of software engineering tools. Although hypotheses are important in debugging, a theory of navigation adds more practical value to our understanding of how programmers debug. Therefore, in this paper, we reconsider how people go about debugging in large collections of source code using a modern programming environment. We present an information foraging theory of debugging that treats programmer navigation during debugging as being analogous to a predator following scent to find prey in the wild. The theory proposes that constructs of scent and topology provide enough information to describe and predict programmer navigation during debugging, without reference to mental states such as hypotheses. We investigate the scope of our theory through an empirical study of 10 professional programmers debugging a real-world open source program. We found that the programmers' verbalizations far more often concerned scent-following than hypotheses. To evaluate the predictiveness of our theory, we created an executable model that predicted programmer navigation behavior more accurately than comparable models that did not consider information scent. Finally, we discuss the implications of our results for enhancing software engineering tools.

This short paper is a summary of my keynote at FSE’16, with accompanying references for follow-up.

The design of programming tools is slow and costly. To ease this process, we developed a design pattern catalog aimed at providing guidance for tool designers. This catalog is grounded in Information Foraging Theory (IFT), which empirical studies have shown to be useful for understanding how developers look for information during development tasks. New design patterns, authored by members of the research community for the catalog, concretely explain how to apply IFT in tool design. In our evaluation, qualitative analyses revealed the community-written design patterns compared well in quality to patterns that we had ourselves published in a smaller, peer-reviewed catalog.

Recent research has reported numerous studies bringing into question the gender inclusiveness of many kinds of software. Inclusiveness of software (gender or otherwise) matters because supporting diversity matters — it is well-known that the more diverse a group of problem-solvers, the higher the quality of the solution. To help software creators identify features within their software that are not gender-inclusive, we recently created a method known as GenderMag. In this paper, we investigate the experience of teams of software professionals using GenderMag to find problems with software they are building. Our results show a high engagement with GenderMag personas — more than twice that of other personas research — and a very high degree of accuracy (93%) most of the time. Finally, our results pinpointed situations that we term “detours” that were especially prone to errors, with teams 6 times more likely to make errors in detours than they did otherwise.

Intelligent systems are gaining in popularity and receiving increased media attention, but little is known about how people actually go about developing them. In this paper, we attempt to fill this gap through a set of field interviews that investigate how people develop intelligent systems that incorporate machine learning algorithms. The developers we interviewed were experienced at working with machine learning algorithms and dealing with the large amounts of data needed to develop intelligent systems. Despite their level of experience, we learned that they struggle to establish a repeatable process. They described problems with each step of the processes they perform, as well as cross-cutting issues that pervade multiple steps of their processes. The unique difficulties that developers like these face seem to point to a need for software engineering advances that address such machine learning systems, and we conclude by discussing this need and some of its implications.

Gender inclusiveness in computing settings is receiving a lot of attention, but one potentially critical factor has mostly been overlooked -- software itself. To help close this gap, we recently created GenderMag, a systematic inspection method to enable software practitioners to evaluate their software for issues of gender-inclusiveness. In this paper, we present the first real-world investigation of software practitioners' ability to identify gender-inclusiveness issues in software they create/maintain using this method. Our investigation was a multiple-case field study of software teams at three major U.S. technology organizations. The results were that, using GenderMag to evaluate software, these software practitioners identified a surprisingly high number of gender-inclusiveness issues: 25% of the software features they evaluated had gender-inclusiveness issues.

Programming languages form the interface between programmers (the users) and the computation that they desire the computer to execute. Although studies exist for some aspects of programming language design (such as conditionals), other aspects have received little or no human factors evaluations. Designers thus have little they can rely on if they want to make new languages highly usable, and users cannot easily chose a language based on usability criteria. This SIG will bring together researchers and practitioners interested in increasing the depth and breadth of studies on the usability of programming languages, and ultimately in improving the usability of future languages. nonPeerReviewed

More people are learning to code than ever, but most learning opportunities do not explicitly teach the problem solving skills necessary to succeed at open-ended programming problems. In this paper, we present a new approach to impart these skills, consisting of: 1) explicit instruction on programming problem solving, which frames coding as a process of translating mental representations of problems and solutions into source code, 2) a method of visualizing and monitoring progression through six problem solving stages, 3) explicit, on-demand prompts for learners to reflect on their strategies when seeking help from instructors, and 4) context-sensitive help embedded in a code editor that reinforces the problem solving instruction. We experimentally evaluated the effects of our intervention across two 2-week web development summer camps with 48 high school students, finding that the intervention increased productivity, independence, programming self-efficacy, metacognitive awareness, and growth mindset. We discuss the implications of these results on learning technologies and classroom instruction.

Foraging among too many variants of the same artifact can be problematic when many of these variants are similar. This situation, which is largely overlooked in the literature, is commonplace in several types of creative tasks, one of which is exploratory programming. In this paper, we investigate how novice programmers forage through similar variants. Based on our results, we propose a refinement to Information Foraging Theory (IFT) to include constructs about variation foraging behavior, and propose refinements to computational models of IFT to better account for foraging among variants.

In recent years, research into gender differences has established that individual differences in how people problem-solve often cluster by gender. Research also shows that these differences have direct implications for software that aims to support users' problem-solving activities, and that much of this software is more supportive of problem-solving processes favored (statistically) more by males than by females. However, there is almost no work considering how software practitioners—such as User Experience (UX) professionals or software developers—can find gender-inclusiveness issues like these in their software. To address this gap, we devised the GenderMag method for evaluating problem-solving software from a gender-inclusiveness perspective. The method includes a set of faceted personas that bring five facets of gender difference research to life, and embeds use of the personas into a concrete process through a gender-specialized Cognitive Walkthrough. Our empirical results show that a variety of practitioners who design software—without needing any background in gender research—were able to use the GenderMag method to find gender-inclusiveness issues in problem-solving software. Our results also show that the issues the practitioners found were real and fixable. This work is the first systematic method to find gender-inclusiveness issues in software, so that practitioners can design and produce problem-solving software that is more usable by everyone.

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article reports the results of a detailed qualitative empirical study of end-user programmers' sensemaking about a spreadsheet's correctness. Using our study's data, we derived a sensemaking model for end-user debugging and categorized participants' activities and verbalizations according to this model, allowing us to investigate how participants went about debugging. Among the results are identification of the prevalence of information foraging during end-user debugging, two successful strategies for traversing the sensemaking model, potential ties to gender differences in the literature, sensemaking sequences leading to debugging progress, and sequences tied with troublesome points in the debugging process. The results also reveal new implications for the design of spreadsheet tools to support end-user programmers' sensemaking during debugging.

Although there has been significant research into gender regarding educational and workplace practices, there has been little awareness of gender differences as they pertain to software tools, such as spreadsheet applications, that try to support end users in problem-solving tasks. Although such software tools are intended to be gender agnostic, we believe that closer examination of this premise is warranted. Therefore, in this paper, we report an end-to-end investigation into gender differences with spreadsheet software. Our results showed gender differences in feature usage, feature-related confidence, and tinkering (playful exploration) with features. Then, drawing implications from these results, we designed and implemented features for our spreadsheet prototype that took the gender differences into account. The results of an evaluation on this prototype showed improvements for both males and females, and also decreased gender differences in some outcome measures, such as confidence. These results are encouraging, but also open new questions for investigation. We also discuss how our results compare to generalization studies performed with a variety of other software platforms and populations.

To help people find a code that they might want to reuse, repositories of end-user code typically sort scripts by number of downloads, ratings, or other information based on prior uses of the code. However, this information is unavailable when the code is new or when it has not yet been reused. Addressing this problem requires identifying reusable code based solely on information that exists when a script is created. To provide such a model for web macro scripts, we identified script traits that might plausibly predict reuse, then used IBM CoScripter repository logs to statistically test how well each corresponded to actual reuse. These tests confirmed that the traits generally did correspond to higher levels of reuse as anticipated. We then developed a machine learning model that uses these traits as features to predict reuse of macros. Evaluating this model on repository logs showed that its accuracy is comparable to that of existing machine learning models for predicting reuse-but with a much simpler structure. Sensitivity analysis revealed that our model is quite robust; its quality is greatly reduced only when parameters are set to such extreme values that the model becomes inordinately selective. Testing the model with individual traits revealed those that provided the best predictions on their own. Based on these results, we outline opportunities for using our model to improve repositories of end-user code.

Can we learn about users' problem-solving strategies by observing their actions? This article introduces a data mining system that extracts complex behavioral patterns from logged user actions to discover users' high-level strategies. Our application domain is an HCI study aimed at revealing users' strategies in an end-user debugging task and understanding how the strategies relate to gender and to success. We cast this problem as a sequential pattern discovery problem, where user strategies are manifested as sequential behavior patterns. Problematically, we found that the patterns discovered by standard data mining algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns as strategies, we examined multiple ways of clustering the patterns into meaningful groups. This collectively led to interesting findings about users' behavior in terms of both gender differences and debugging success. These common behavioral patterns were novel HCI findings about differences in males' and females' behavior with software, and were verified by a parallel study with an independent data set on strategies. As a research endeavor into the interpretability issues faced by data mining techniques, our work also highlights important research directions for making data mining more accessible to non-data-mining experts.

End-user programming has become ubiquitous; so much so that there are more end-user programmers today than there are professional programmers. End-user programming empowers—but to do what? Make bad decisions based on bad programs? Enter software engineering’s focus on quality. Considering software quality is necessary, because there is ample evidence that the programs end users create are filled with expensive errors. In this paper, we consider what happens when we add considerations of software quality to end-user programming environments, going beyond the “create a program” aspect of end-user programming. We describe a philosophy of software engineering for end users, and then survey several projects in this area. A basic premise is that end-user software engineering can only succeed to the extent that it respects that the user probably has little expertise or even interest in software engineering.

Rule-based programming systems can be fragile because they force the user to account for all logical alternatives. If an unconsidered case does arise during execution, program behavior falls through the cracks into unspecified behavior. We investigate rule-based, end-user strategy programming by introducing our Interactive Football Playbook-a domain specific, end-user programming environment to allow American football coaches to create animated football scenarios by associating strategy information with virtual football players. We address the problem of rule explosion through ''rule bending'' to support a minimalist, scaffolding-driven programming environment. Additionally, we introduce visual language representations for logical and sequential ''and'' to mitigate end-user confusion with the semantic meaning of these ''and'' constructs.

Many systems are designed to help novices who want to learn programming, but few support those who are not interested in learning (more) programming. This paper targets the subset of end-user programmers (EUPs) in this category. We present a set of principles on how to help EUPs like this learn just a little when they need to overcome a barrier. We then instantiate the principles in a prototype and empirically investigate the principles in two studies: a formative think-aloud study and a pair of summer camps attended by 42 teens. Among the surprising results were the complementary roles of implicitly actionable hints versus explicitly actionable hints, and the importance of both context-free and context-sensitive availability. Under these principles, the camp participants required significantly less in-person help than in a previous camp to learn the same amount of material in the same amount of time.

Developers performing maintenance activities must balance their efforts to learn the code vs. their efforts to actually change it. This balancing act is consistent with the “production bias” that, according to Carroll's minimalist learning theory, generally affects software users during everyday tasks. This suggests that developers' focus on efficiency should have marked effects on how they forage for the information they think they need to fix bugs. To investigate how developers balance fixing versus learning during debugging, we conducted the first empirical investigation of the interplay between production bias and information foraging. Our theory-based study involved 11 participants: half tasked with fixing a bug, and half tasked with learning enough to help someone else fix it. Despite the subtlety of difference between their tasks, participants foraged remarkably differently-making foraging decisions from different types of “patches,” with different types of information, and succeeding with different foraging tactics.

Recent research has shown that some software that is intended to be gender-neutral is not, in fact, equally inclusive to males and females. But little is known about how to design software in a gender-aware fashion, and existing research on gender differences relevant to software design is scattered across at least five different academic fields (e.g., psychology, computer science, education, communications, and women's studies). This research SIG will bring together female and male academics, industry researchers, and practitioners with three goals in mind: (1) to build community across research/practice boundaries; (2) to pool our knowledge on promising practices for design and evaluation of software from a gender perspective; and (3) to begin to build a shared, on-line research and literature base to support solid, well-informed progress on this important issue.

How can end users efficiently influence the predictions that machine learning systems make on their behalf? This paper presents Explanatory Debugging, an approach in which the system explains to users how it made each of its predictions, and the user then explains any necessary corrections back to the learning system. We present the principles underlying this approach and a prototype instantiating it. An empirical evaluation shows that Explanatory Debugging increased participants' understanding of the learning system by 52% and allowed participants to correct its mistakes up to twice as efficiently as participants using a traditional learning system.

The paradigm of end-user development enables ordinary end-users of computer systems to engage in the modification, extension and even creation of software artifacts. Technology, organization and context are all important aspects that influence end-users' decisions to engage in end-user development. With this workshop, we invite researchers and practitioners to discuss and exchange their experiences on the role of these aspects for end-user development across various domains.

We use the term gender HCI (human-computer interaction) to refer to research into how software relates to gender differences. Our particular focus is on how gender-neutral software works, not on gender-oriented content. Specifically, we have concentrated on software aimed at supporting everyday users doing problem solving. Several of the hypotheses we developed in the first step of our investigative method predict that in a software environment with problem-solving features, gender differences will have a significant impact on the adoption and use of these features, due at least in part to differences related to risk perception and confidence. To maintain its creative edge, an organization must encourage a diversity of ideas