Search

Gender inclusivity in software is gaining attention from researchers and practitioners, with some seeing it as a nonfunctional requirement. To investigate how gender inclusivity can be incorporated into creating software, we gathered data during periods ranging from 5 months to 3.5 years from 10 software teams that used the Gender Inclusiveness Magnifier (GenderMag) to achieve the gender inclusivity quality attribute. GenderMag is a method for detecting and fixing gender inclusivity issues in software. In this article, we summarize several practices the teams devised and pitfalls they encountered.

Motivation: Although CS Education researchers and practitioners have found ways to improve CS classroom inclusivity, few researchers have considered inclusivity of online CS education. We are interested in two such improvements in online CS education—besides being inclusive to each other, online CS students also need to be able to create inclusive technology. Objectives: We have begun developing a new approach that we term “embedded inclusive design” to address both of these goals. The essence of the approach is to integrate elements of inclusive design education into mainstream CS coursework. This paper presents three curricular interventions we have developed in this approach and empirically investigates their efficacy in online CS post-baccalaureate education. Our research questions were: How do these three curricular interventions affect (RQ1) the climate among online CS students and (RQ2) how online CS students honor the diversity of their users in the tech they create? Method: To answer these research questions, we implemented the curricular interventions in four asynchronous online CS classes across two CS courses within Oregon State University’s Ecampus and conducted an action research study to investigate the impacts. Results: Online CS students who experienced these interventions reported feeling more included in the major than they had before, reported positive impacts on their team dynamics, increased their interest in accommodating diverse users, and created more inclusive technology designs than they had before. Discussion: These results provide encouraging evidence that embedding elements of inclusive design into mainstream CS coursework, via the interventions presented here, can increase both online CS students’ inclusivity toward one another and the inclusivity of the technology these future CS practitioners create.

Web-active end-user programmers squander much of their time foraging for bugs and related information in mashup programming environments as well as on the web. To analyze this foraging behavior while debugging, we utilize an Information Foraging Theory perspective. Information Foraging Theory models the human (predator) behavior to forage for specific information (prey) in the webpages or programming IDEs (patches) by following the information features (cues) in the environment. We qualitatively studied the debugging behavior of 16 web-active end users. Our results show that end-user programmers spend substantial amounts (73%) of their time just foraging. Further, our study reveals new cue types and foraging strategies framed in terms of Information Foraging Theory, and it uncovers which of these helped end-user programmers succeed in their debugging efforts.

Does your college/university say one of its core values is diversity, equity, and inclusion? If so, is the IT they are using equitable and inclusive for usage by diverse populations of users? The evidence suggests “no” — and in this talk, we will consider how to address this problem by answering the following questions: How can IT professionals assess whether their IT supports diverse users? And if they find problems, how can they fix them? Although there are empirical processes that can be used to find “inclusivity bugs” piecemeal in software, web apps, websites, and so on, what is often needed is a systematic inspection method to assess IT's support for diverse populations. To help fill this gap, we developed GenderMag, a method for finding and fixing “gender inclusivity bugs” — gender biases in IT interfaces and workflows. We then introduced InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for other dimensions of diversity. In this talk, we present the latest GenderMag results, what OSU's IT professionals are doing with it, and provide a glimpse into the future by briefly introducing InclusiveMag and our early experiences with it.

How can user interface and user experience (UI/UX) professionals assess whether their software supports diverse users? And if they find problems, how can they fix them? We begin this keynote address with a summary of GenderMag, a systematic inspection method for finding and fixing "gender inclusivity bugs---biases against different genders in software interfaces and workflows. We then show what UI/UX professionals are doing with it in the real world, from their bias finds & fixes to their practices & pitfalls in using it. Finally, we present InclusiveMag, a meta-method that can be used by HCI researchers to generate systematic inclusiveness methods for other dimensions of diversity.

Explainable AI (XAI) has started experiencing explosive growth, echoing the explosive growth that has preceded it of AI becoming used for practical purposes that impact the general public. This spread of AI into the world outside of research labs brings with it pressures and requirements that many of us have perhaps not thought about deeply enough. In this keynote address, I will explain why I think we have a very long way to go. One way to characterize our current state is that we're doing "fairly well", doing some explaining of some things. In a sense, this is reasonable: the XAI field is young, and still finding its way. However, moving forward demands progress in (at least) three areas. (1) How we go about XAI research: Explainable AI cannot succeed if the only research foundations brought to bear on it are AI foundations. Likewise, it cannot succeed if the only foundations used are from psychology, education, etc. Thus, a challenge for our emerging field is how to conduct XAI research in a truly effective multi-disciplinary fashion, that is based on an integration of foundations behind what we can make AI algorithms do, with solid, well-founded principles of explaining the complex ideas behind the algorithms to real people. Fortunately, a few researchers have started to build such foundations. (2) What we can succeed at explaining: So far, we as a field are doing a certain amount of cherry picking as to what we explain. We tend to choose what to explain by what we can figure out how to explain---but we are leaving too much out. One urgent case in point is the societal and legal need to explain fairness properties of AI systems. The above challenges are important, but the field is already becoming aware of them. Thus, this keynote will focus mostly on the third challenge, namely: (3) Who we can explain to. Who are the people we've even tried to explain AI to, so far? What are the societal implications of who we explain to well and who we do not? Our field has not even begun to consider this question. In this keynote I'll discuss why we have to explain to populations to whom we've given little thought---diverse people in many dimensions, including gender diversity, cognitive diversity, and age diversity. Addressing all of these challenges is necessary before we can claim to explain AI fairly and well.

How can software practitioners assess whether their software supports diverse users? Although there are empirical processes that can be used to find "inclusivity bugs" piecemeal, what is often needed is a systematic inspection method to assess soft-ware's support for diverse populations. To help fill this gap, this paper introduces InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for a particular dimension of diversity. We then present a multi-case study covering eight diversity dimensions, of eight teams' experiences applying InclusiveMag to eight under-served populations and their "mainstream" counterparts.

Background Health communication tools like film are capable of reducing health disparities and could be effective in addressing negative illness perceptions of MS in Hispanics/Latinx. Objective To test the feasibility of using a culturally appropriate short narrative film to examine illness perceptions overtime and attitudes in Hispanics/Latinx affected with MS. Methods Participants were assigned to view a short narrative film (n = 130) or not (n = 106). The Brief Illness Perception Questionnaire (BIPQ) was used to examine illness perceptions at baseline, one and three months. Focus groups were conducted at 6 months. Measures of sociocultural integration were obtained. Individual group BIPQ domains were evaluated over time using paired sample t-test. Multivariate linear regression was used to examine predictors of BIPQ change. Results A more positive perception of treatment (p Conclusion Our results provide support that a short narrative film of MS in Hispanics/Latinx is a feasible intervention to change perceptions of MS to a more positive view.

Personas often aim to improve product designers' ability to "see through the eyes of" target users through the empathy personas can inspire - but personas are also known to promote stereotyping. This tension can be particularly problematic when personas (who, of course as "people" have genders) are used to promote gender inclusiveness - because reinforcing stereotypical perceptions can run counter to gender inclusiveness. In this paper we explicitly investigate this tension through a new approach to personas: one that includes multiple photos (of males and females) for a single persona. We compared this approach to an identical persona with only one photo using a controlled laboratory study and an eye-tracking study. Our goal was to answer the following question: is it possible for personas to encourage product designers to engage with personas while at the same avoiding promoting gender stereotyping? Our results are encouraging about the use of personas with multiple pictures as a way to expand participants' consideration of multiple genders without reducing their engagement with the persona.

How should an AI-based explanation system explain an agent's complex behavior to ordinary end users who have no background in AI? Answering this question is an active research area, for if an AI-based explanation system could effectively explain intelligent agents' behavior, it could enable the end users to understand, assess, and appropriately trust (or distrust) the agents attempting to help them. To provide insights into this question, we turned to human expert explainers in the real-time strategy domain, "shoutcaster", to understand (1) how they foraged in an evolving strategy game in real time, (2) how they assessed the players' behaviors, and (3) how they constructed pertinent and timely explanations out of their insights and delivered them to their audience. The results provided insights into shoutcasters' foraging strategies for gleaning information necessary to assess and explain the players; a characterization of the types of implicit questions shoutcasters answered; and implications for creating explanations by using the patterns, Comment: 12 pages, 11 figures, submitted to CHI 2017

One area of research in the end-user development area is known as end-user software engineering (EUSE). Research in EUSE aims to invent new kinds of technologies that collaborate with end users to improve the quality of their software. EUSE has become an active research area since its birth in the early 2000s, with a large body of literature upon which EUSE researchers can build. However, building upon these works can be difficult when projects lack connections due to an absence of cross-cutting foundations to tie them together. In this chapter, we advocate for stronger theory foundations and show the advantages through three theory-oriented projects: (1) the Explanatory Debugging approach, to help end users debug their intelligent assistants; (2) the GenderMag method, which identifies problems with gender inclusiveness in EUSE tools and other software; and (3) the Idea Garden approach, to help end users to help themselves in overcoming programming barriers. In each of these examples, we show how having a theoretical foundation facilitated generalizing beyond individual tools to the production of general methods and principles for other researchers to directly draw upon in their own works.

Theories of human behavior are an important but largely untapped resource for software engineering research. They facilitate understanding of human developers’ needs and activities, and thus can serve as a valuable resource to researchers designing software engineering tools. Furthermore, theories abstract beyond specific methods and tools to fundamental principles that can be applied to new situations. Toward filling this gap, we investigate the applicability and utility of Information Foraging Theory (IFT) for understanding information-intensive software engineering tasks, drawing upon literature in three areas: debugging, refactoring, and reuse. In particular, we focus on software engineering tools that aim to support information-intensive activities, that is, activities in which developers spend time seeking information. Regarding applicability, we consider whether and how the mathematical equations within IFT can be used to explain why certain existing tools have proven empirically successful at helping software engineers. Regarding utility, we applied an IFT perspective to identify recurring design patterns in these successful tools, and consider what opportunities for future research are revealed by our IFT perspective.

Many theories of human debugging rely on complex mental constructs that offer little practical advice to builders of software engineering tools. Although hypotheses are important in debugging, a theory of navigation adds more practical value to our understanding of how programmers debug. Therefore, in this paper, we reconsider how people go about debugging in large collections of source code using a modern programming environment. We present an information foraging theory of debugging that treats programmer navigation during debugging as being analogous to a predator following scent to find prey in the wild. The theory proposes that constructs of scent and topology provide enough information to describe and predict programmer navigation during debugging, without reference to mental states such as hypotheses. We investigate the scope of our theory through an empirical study of 10 professional programmers debugging a real-world open source program. We found that the programmers' verbalizations far more often concerned scent-following than hypotheses. To evaluate the predictiveness of our theory, we created an executable model that predicted programmer navigation behavior more accurately than comparable models that did not consider information scent. Finally, we discuss the implications of our results for enhancing software engineering tools.

Empirical studies have revealed that software developers spend 35%–50% of their time navigating through source code during development activities, yet fundamental questions remain: Are these percentages too high, or simply inherent in the nature of software development? Are there factors that somehow determine a lower bound on how effectively developers can navigate a given information space? Answering questions like these requires a theory that captures the core of developers' navigation decisions. Therefore, we use the central proposition of Information Foraging Theory to investigate developers' ability to predict the value and cost of their navigation decisions. Our results showed that over 50% of developers' navigation choices produced less value than they had predicted and nearly 40% cost more than they had predicted. We used those results to guide a literature analysis, to investigate the extent to which these challenges are met by current research efforts, revealing a new area of inquiry with a rich and crosscutting set of research challenges and open problems.

More people are learning to code than ever, but most learning opportunities do not explicitly teach the problem solving skills necessary to succeed at open-ended programming problems. In this paper, we present a new approach to impart these skills, consisting of: 1) explicit instruction on programming problem solving, which frames coding as a process of translating mental representations of problems and solutions into source code, 2) a method of visualizing and monitoring progression through six problem solving stages, 3) explicit, on-demand prompts for learners to reflect on their strategies when seeking help from instructors, and 4) context-sensitive help embedded in a code editor that reinforces the problem solving instruction. We experimentally evaluated the effects of our intervention across two 2-week web development summer camps with 48 high school students, finding that the intervention increased productivity, independence, programming self-efficacy, metacognitive awareness, and growth mindset. We discuss the implications of these results on learning technologies and classroom instruction.

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article reports the results of a detailed qualitative empirical study of end-user programmers' sensemaking about a spreadsheet's correctness. Using our study's data, we derived a sensemaking model for end-user debugging and categorized participants' activities and verbalizations according to this model, allowing us to investigate how participants went about debugging. Among the results are identification of the prevalence of information foraging during end-user debugging, two successful strategies for traversing the sensemaking model, potential ties to gender differences in the literature, sensemaking sequences leading to debugging progress, and sequences tied with troublesome points in the debugging process. The results also reveal new implications for the design of spreadsheet tools to support end-user programmers' sensemaking during debugging.

To help people find a code that they might want to reuse, repositories of end-user code typically sort scripts by number of downloads, ratings, or other information based on prior uses of the code. However, this information is unavailable when the code is new or when it has not yet been reused. Addressing this problem requires identifying reusable code based solely on information that exists when a script is created. To provide such a model for web macro scripts, we identified script traits that might plausibly predict reuse, then used IBM CoScripter repository logs to statistically test how well each corresponded to actual reuse. These tests confirmed that the traits generally did correspond to higher levels of reuse as anticipated. We then developed a machine learning model that uses these traits as features to predict reuse of macros. Evaluating this model on repository logs showed that its accuracy is comparable to that of existing machine learning models for predicting reuse-but with a much simpler structure. Sensitivity analysis revealed that our model is quite robust; its quality is greatly reduced only when parameters are set to such extreme values that the model becomes inordinately selective. Testing the model with individual traits revealed those that provided the best predictions on their own. Based on these results, we outline opportunities for using our model to improve repositories of end-user code.

Can we learn about users' problem-solving strategies by observing their actions? This article introduces a data mining system that extracts complex behavioral patterns from logged user actions to discover users' high-level strategies. Our application domain is an HCI study aimed at revealing users' strategies in an end-user debugging task and understanding how the strategies relate to gender and to success. We cast this problem as a sequential pattern discovery problem, where user strategies are manifested as sequential behavior patterns. Problematically, we found that the patterns discovered by standard data mining algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns as strategies, we examined multiple ways of clustering the patterns into meaningful groups. This collectively led to interesting findings about users' behavior in terms of both gender differences and debugging success. These common behavioral patterns were novel HCI findings about differences in males' and females' behavior with software, and were verified by a parallel study with an independent data set on strategies. As a research endeavor into the interpretability issues faced by data mining techniques, our work also highlights important research directions for making data mining more accessible to non-data-mining experts.

Developers performing maintenance activities must balance their efforts to learn the code vs. their efforts to actually change it. This balancing act is consistent with the “production bias” that, according to Carroll's minimalist learning theory, generally affects software users during everyday tasks. This suggests that developers' focus on efficiency should have marked effects on how they forage for the information they think they need to fix bugs. To investigate how developers balance fixing versus learning during debugging, we conducted the first empirical investigation of the interplay between production bias and information foraging. Our theory-based study involved 11 participants: half tasked with fixing a bug, and half tasked with learning enough to help someone else fix it. Despite the subtlety of difference between their tasks, participants foraged remarkably differently-making foraging decisions from different types of “patches,” with different types of information, and succeeding with different foraging tactics.

How can end users efficiently influence the predictions that machine learning systems make on their behalf? This paper presents Explanatory Debugging, an approach in which the system explains to users how it made each of its predictions, and the user then explains any necessary corrections back to the learning system. We present the principles underlying this approach and a prototype instantiating it. An empirical evaluation shows that Explanatory Debugging increased participants' understanding of the learning system by 52% and allowed participants to correct its mistakes up to twice as efficiently as participants using a traditional learning system.

We use the term gender HCI (human-computer interaction) to refer to research into how software relates to gender differences. Our particular focus is on how gender-neutral software works, not on gender-oriented content. Specifically, we have concentrated on software aimed at supporting everyday users doing problem solving. Several of the hypotheses we developed in the first step of our investigative method predict that in a software environment with problem-solving features, gender differences will have a significant impact on the adoption and use of these features, due at least in part to differences related to risk perception and confidence. To maintain its creative edge, an organization must encourage a diversity of ideas

End-user programmers develop more software than any other group of programmers, using software authoring devices such as multimedia simulation builders, e-mail filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been only a little research on finding ways to help these programmers with the dependability of the software they create. We have been working to address this problem in several ways, one of which includes supporting end-user debugging activities through interactive fault localization techniques. This paper investigates fault localization techniques in the spreadsheet domain, the most common type of end-user programming environment. We investigate a technique previously described in the research literature and two new techniques. We present the results of an empirical study to examine the impact of two individual factors on the effectiveness of fault localization techniques. Our results reveal several insights into the contributions such techniques can make to the end-user debugging process and highlight key issues of interest to researchers and practitioners who may design and evaluate future fault localization techniques.

Extending our previous work [T. Robertson, S. Prabhakararao, M. Burnett, C. Cook, J. Ruthruff, L. Beckwith, A. Phalgune, Impact of interruption style on end-user debugging, ACM Conference on Human Factors in Computing Systems (2004)], we delve deeper into the question of which interruption style best supports end-user debugging. Previously, we found no advantages of immediate-style interruptions (which force the user to divert attention to the interruption at hand) over negotiated-style interruptions (which notify users without actually preventing them from working) in supporting end-user debugging. In this study, we altered our negotiated-style interruptions [A. Wilson, M. Burnett, L. Beckwith, O. Granatir, L. Casburn, C. Cook, M. Durham, G. Rothermel, Harnessing curiosity to increase correctness in end-user programming, Proceedings of the CHI 2003 (2003), 305-312] (which were shown to help end-user debuggers learn about and use debugging features of our programming language) such that they were more intense (larger, blinking, and/or accompanied by text).

End-user programmers are writing an unprecedented number of programs, primarily using languages and environments that incorporate a number of interactive and visual programming techniques. To help these users debug these programs, we have developed an entirely visual, interactive approach to fault localization. This paper presents the approach. We also present the results of a think-aloud study that examined interactive, human-centric issues that arise in end-user debugging using a fault localization approach. Our results provide insights into the contributions such approaches can make to the end-user debugging process.

Although there are many systems designed to engage people in programming, few explicitly teach the subject, expecting learners to acquire the necessary skills on their own as they create programs from scratch. We present a principled approach to teach programming using a debugging game called Gidget, which was created using a unique set of seven design principles. A total of 44 teens played it via a lab study and two summer camps. Principle by principle, the results revealed strengths, problems, and open questions for the seven principles. Taken together, the results were very encouraging: learners were able to program with conditionals, loops, and other programming concepts after using the game for just 5 hours.

End-user software engineering (EUSE) is a research area that aims to invent new kinds of technologies that collaborate with end users to improve the quality of their software. The practice that EUSE research aims to support is end users using new tools and methods to improve the quality of the software that they and other end users have created. There is a need for this outcome because research shows both that the number of end users creating their own software greatly exceeds the number of professional software developers, and that the software they create is riddled with errors. In this paper, we describe the present state of EUSE, and challenges in moving forward toward a bright future. We show how the future of EUSE may become over-siloed, restricting future researchers' vision of what can be achieved. We then show that focusing on the in-the-moment intents of end-user developers can be used to derive a number of promising directions forward for EUSE researchers, and how theories can help us further de-silo future EUSE research. Finally, we discuss how overcoming challenges for the future of end-user software engineering may also bring direct benefits to the future of "classic" software engineering.

Spreadsheet languages, which include commercial spreadsheets and various research systems, have proven to be flexible tools in many domain specific settings. Research shows, however, that spreadsheets often contain faults. We would like to provide at least some of the benefits of formal testing and debugging methodologies to spreadsheet developers. This paper presents an integrated testing and debugging methodology for spreadsheets. To accommodate the modeless and incremental development, testing and debugging activities that occur during spreadsheet creation, our methodology is tightly integrated into the spreadsheet environment. To accommodate the users of spreadsheet languages, we provide an interface to our methodology that does not require an understanding of testing and debugging theory, and that takes advantage of the immediate visual feedback that is characteristic of the spreadsheet paradigm.

Research is emerging on how end users can correct mistakes their intelligent agents make, but before users can correctly "debug" an intelligent agent, they need some degree of understanding of how it works. In this paper we consider ways intelligent agents should explain themselves to end users, especially focusing on how the soundness and completeness of the explanations impacts the fidelity of end users' mental models. Our findings suggest that completeness is more important than soundness: increasing completeness via certain information types helped participants' mental models and, surprisingly, their perception of the cost/benefit tradeoff of attending to the explanations. We also found that oversimplification, as per many commercial agents, can be a problem: when soundness was very low, participants experienced more mental demand and lost trust in the explanations, thereby reducing the likelihood that users will pay attention to such explanations at all.

In this paper, we investigate how a debugging environment should support a population doing work at the core of HCI research: cognitive modelers. In conducting this investigation, we extended the Natural Programming methodology (a user-centered design method for HCI researchers of programming environments), to add an explicit method for mapping the outcomes of NP's empirical investigations to a language design. This provided us with a concrete way to make the design leap from empirical assessment of users' needs to a language. The contributions of our work are therefore: (1) empirical evidence about the content and sequence of cognitive modelers' information needs when debugging, (2) a new, empirically derived, design specification for a debugging interaction language for cognitive modelers, and (3) an initial case study of our "Natural Programming Plus" methodology.

Information Foraging Theory (IFT) has established itself as an important theory to explain how people seek information, but most work has focused more on the theory itself than on how best to apply it. In this paper, we investigate how to apply a reactive variant of IFT (Reactive IFT) to design IFT-based tools, with a special focus on such tools for ill-structured problems. Toward this end, we designed and implemented a variety of recommender algorithms to empirically investigate how to help people with the ill-structured problem of finding where to look for information while debugging source code. We varied the algorithms based on scent type supported (words alone vs. words + code structure), and based on use of foraging momentum to estimate rapidity of foragers' goal changes. Our empirical results showed that (1) using both words and code structure significantly improved the ability of the algorithms to recommend where software developers should look for information; (2) participants used recommendations to discover new places in the code and also as shortcuts to navigate to known places; and (3) low-momentum recommendations were significantly more useful than high-momentum recommendations, suggesting rapid and numerous goal changes in this type of setting. Overall, our contributions include two new recommendation algorithms, empirical evidence about when and why participants found IFT-based recommendations useful, and implications for the design of tools based on Reactive IFT.

What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender systems are gaining widespread use, potentially creating a need for end users to understand how these systems operate in order to fix their agent's personalized behavior. This paper explores the effects of mental model soundness on such personalization by providing structural knowledge of a music recommender system in an empirical study. Our findings show that participants were able to quickly build sound mental models of the recommender system's reasoning, and that participants who most improved their mental models during the study were significantly more likely to make the recommender operate to their satisfaction. These results suggest that by helping end users understand a system's reasoning, intelligent agents may elicit more and better feedback, thus more closely aligning their output with each user's intentions.

Machine learning techniques are increasingly used in intelligent assistants , that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user “debugging” of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these “why” questions, and allows users to interactively change these answers to debug the assistant's current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.

Despite recent advances in supporting end-user programmers, empirical studies continue to report barriers that end users experience in problem solving with programming environments. We hypothesize that an important barrier that still needs to be overcome is the lack of support for nurturing end-user programmers' ideas on how a program should be written or on how to solve programming difficulties. Therefore, in this paper, we present a qualitative empirical investigation and triangulate the results with theories from problem solving and creativity. Moreover, we explore design opportunities and a design space for “idea gardening”, a new approach to nurturing end-user programmers' ideas and to helping them gradually gain expertise as they overcome barriers. Our results suggest that nurturing end-user programmers' ideas is a fertile area for research with an interesting, multidimensional design space.

Software developers frequently need to perform code maintenance tasks, but doing so requires time-consuming navigation through code. A variety of tools are aimed at easing this navigation by using models to identify places in the code that a developer might want to visit, and then providing shortcuts so that the developer can quickly navigate to those locations. To date, however, only a few of these models have been compared head-to-head to assess their predictive accuracy. In particular, we do not know which models are most accurate overall, which are accurate only in certain circumstances, and whether combining models could enhance accuracy. Therefore, we have conducted an empirical study to evaluate the accuracy of a broad range of models for predicting many different kinds of code navigations in sample maintenance tasks. Overall, we found that models tended to perform best if they took into account how recently a developer has viewed pieces of the code, and if models took into account the spatial proximity of methods within the code. We also found that the accuracy of single-factor models can be improved by combining factors, using a spreading-activation based approach, to produce multi-factor models. Based on these results, we offer concrete guidance about how these models could be used to provide enhanced software development tools that ease the difficulty of navigating through code.

In recent years, systems have emerged that enable end users to “mash” together existing web services to build new web sites. However, little is known about how well end users succeed at building such mashups, or what they do if they do not succeed at their first attempt. To help fill this gap, we took a fresh look, from a debugging perspective, at the approaches of end users as they attempted to create mashups. Our results reveal the end users’ debugging strategies and strategy barriers, the gender differences between the debugging strategies males and females followed and the features they used, and finally how their debugging successes and difficulties interacted with their design behaviors.

Are the abstractions that scientific modelers use to build their models in a modeling language the same abstractions they use to evaluate the correctness of their models? The extent to which such differences exist seems likely to correspond to additional effort of modelers in determining whether their models work as intended. In this paper, we therefore investigate the distinction between "programming abstractions" and "evaluation abstractions". As the basis of our investigation, we conducted a case study on cognitive modeling. We report modelers' evaluation abstractions, and the lengths they went to in evaluating their models. From these results, we derive design implications for several categories of persistent, first-class evaluation abstractions in future debugging tools for modelers.

Programming has recently become more common among ordinary end users of computer systems. We believe that these end-user programmers are not just coders but also designers, in that they interlace making design decisions with coding rather than treating them as two separate phases. To better understand and provide support for the programming and design needs of end users, we propose a design theory-based approach to look at end-user programming. Toward this end, we conducted a think-aloud study with ten end users creating a web mashup. By analyzing users' verbal and behavioral data using Schon's reflection-in-action design model and the notion of ideations from creativity literature, we discovered insights into end-user programmers' problem-solving attempts, successes, and obstacles, with accompanying implications for the design of end-user programming environments for mashups. The contribution of our work is three-fold: 1) the methodology of using a design lens to view programming, 2) evidence, through insights gained, of the usefulness of this approach, and 3) the implications themselves.

End-user programmers' code is notoriously buggy. This problem is amplified by the increasing complexity of end users' programs. To help end users catch errors early and reliably, we employ a novel approach for the design of end-user debugging tools: a focus on supporting end users' effective debugging strategies. This paper makes two contributions. We first demonstrate the potential of a strategy-centric approach to tool design by presenting StratCel, an add-in for Excel. Second, we show the benefits of this design approach: participants using StratCel found twice as many bugs as participants using standard Excel, they fixed four times as many bugs, and all this in only a small fraction of the time. Other contributions included: a boost in novices' debugging performance near experienced participants' improved levels, validated design guidelines, a discussion of the generalizability of this approach, and several opportunities for future research.

End-user programmers may not be aware of many software engineering practices that would add greater discipline to their efforts, and even if they are aware of them, these practices may seem too costly (in terms of time) to use. Without taking advantage of at least some of these practices, the software these end users create seems likely to continue to be less reliable than it could be. We are working on several ways of lowering both the perceived and actual costs of systematic software engineering practices, and on making their benefits more visible and immediate. Our approach is to leverage the user's cognitive effort through the use of distributed cognition, in which the system and user collaboratively work systematically to reason about the program the end user is creating. This paper demonstrates this concept with a few of our past efforts, and then presents three of our current efforts in this direction.

To provide evidence-based guidance for women and their health care providers on the management of missed or delayed hormonal contraceptive doses in order to prevent unintended pregnancy.Medline, PubMed, and the Cochrane Database were searched for articles published in English, from 1974 to 2007, about hormonal contraceptive methods that are available in Canada and that may be missed or delayed. Relevant publications and position papers from appropriate reproductive health and family planning organizations were also reviewed. The quality of evidence is rated using the criteria developed by the Canadian Task Force on Preventive Health Care.This committee opinion will help health care providers offer clear information to women who have not been adherent in using hormonal contraception with the purpose of preventing unintended pregnancy.The Society of Obstetricians and Gynaecologists of Canada. SUMMARY STATEMENTS: 1. Instructions for what women should do when they miss hormonal contraception have been complex and women do not understand them correctly. (I) 2. The highest risk of ovulation occurs when the hormone-free interval is prolonged for more than seven days, either by delaying the start of combined hormonal contraceptives or by missing active hormone doses during the first or third weeks of combined oral contraceptives. (II) Ovulation rarely occurs after seven consecutive days of combined oral contraceptive use. (II) RECOMMENDATIONS: 1. Health care providers should give clear, simple instructions, both written and oral, on missed hormonal contraceptive pills as part of contraceptive counselling. (III-A) 2. Health care providers should provide women with telephone/electronic resources for reference in the event of missed or delayed hormonal contraceptives. (III-A) 3. In order to avoid an increased risk of unintended pregnancy, the hormone-free interval should not exceed seven days in combined hormonal contraceptive users. (II-A) 4. Back-up contraception should be used after one missed dose in the first week of hormones until seven consecutive days of correct hormone use are established. In the case of missed combined hormonal contraceptives in the second or third week of hormones, the hormone-free interval should be eliminated for that cycle. (III-A) 5. Emergency contraception and back-up contraception may be required in some instances of missed hormonal contraceptives, in particular when the hormone-free interval has been extended for more than seven days. (III-A) 6. Back-up contraception should be used when three or more consecutive doses/days of combined hormonal contraceptives are missed in the second and third week until seven consecutive days of correct hormone use are established. For practical reasons, the scheduled hormone-free interval should be eliminated in these cases. (II-A) 7. Emergency contraception is rarely indicated for missed combined hormonal contraceptives in the second or third week of the cycle unless there are repeated omissions or failure to institute back-up contraception after the missed doses. In cases of repeated omissions of combined hormonal contraceptives, emergency contraception may be required, and back-up contraception should be used. Health care professionals should counsel women in these situations on alternative methods of contraception that do not demand such stringent compliance. (III-A).

Little research has addressed IT professionals' script debugging strategies, or considered whether there may be gender differences in these strategies. What strategies do male and female scripters use and what kinds of mechanisms do they employ to successfully fix bugs? Also, are scripters' debugging strategies similar to or different from those of spreadsheet debuggers? Without the answers to these questions, tool designers do not have a target to aim at for supporting how male and female scripters want to go about debugging. We conducted a think-aloud study to bridge this gap. Our results include (1) a generalized understanding of debugging strategies used by spreadsheet users and scripters, (2) identification of the multiple mechanisms scripters employed to carry out the strategies, and (3) detailed examples of how these debugging strategies were employed by males and females to successfully fix bugs.

The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may need to be debugged. Providing ways for the user to debug it matters, because without the ability to fix errors users may find that the learned program's errors are too damaging for them to be able to trust such programs. We present a new approach to enable end users to debug a learned program. We then use an early prototype of our new approach to conduct a formative study to determine where and when debugging issues arise, both in general and also separately for males and females. The results suggest opportunities to make machine-learned programs more effective tools.

Recommendations by software repositories depend on explicit or implicit models for evaluating the quality and relevance of components for programming tasks. As a step toward creating such a model for evaluating end-user web macro scripts, we have identified script characteristics that correspond to the likelihood of script reuse. For example, the likelihood of reuse increases with the number of variables and comments in the script, the number of online forum postings by the script’s author, and the presence of popular keywords in the script’s source code. We discuss possible applications of our results for new recommendation features.

Recent research has begun to report that female end-user programmers are often more reluctant than males to employ features that are useful for testing and debugging. These earlier findings suggest that, unless such features can be changed in some appropriate way, there are likely to be important gender differences in end-user programmerspsila benefits from these features. In this paper, we compare end-user programmerspsila feature usage in an environment that supports end-user debugging, against an extension of the same environment with two features designed to help ameliorate the effects of low self-efficacy. Our results show ways in which these features affect female versus male enduser programmerspsila self-efficacy, attitudes, usage of testing and debugging features, and performance.

Little is known about the strategies end-user programmers use in debugging their programs, and even less is known about gender differences that may exist in these strategies. Without this type of information, designers of end-user programming systems cannot know the "target" at which to aim, if they are to support male and female end-user programmers. We present a study investigating this issue. We asked end-user programmers to debug spreadsheets and to describe their debugging strategies. Using mixed methods, we analyzed their strategies and looked for relationships among participants' strategy choices, gender, and debugging success. Our results indicate that males and females debug in quite different ways, that opportunities for improving support for end-user debugging strategies for both genders are abundant, and that tools currently available to end-user debuggers may be especially deficient in supporting debugging strategies used by females.

During maintenance, professional developers generate and test many hypotheses about program behavior, but they also spend much of their time navigating among classes and methods. Little is known, however, about how professional developers navigate source code and the extent to which their hypotheses relate to their navigation. A lack of understanding of these issues is a barrier to tools aiming to reduce the large fraction of time developers spend navigating source code. In this paper, we report on a study that makes use of information foraging theory to investigate how professional developers navigate source code during maintenance. Our results showed that information foraging theory was a significant predictor of the developers' maintenance behavior, and suggest how tools used during maintenance can build upon this result, simply by adding word analysis to their reasoning systems.

There has been little research into how end-user programming environments can provide explanations that could fill a critical information gap for end-user debuggers - help with debugging strategy. To address this need, we designed and prototyped a video-based approach for explaining debugging strategy, and accompanied it with a text-only approach. We then conducted a qualitative empirical study with end-user debuggers. The results reveal the influences of the explanations on end-user debuggers' decision making, how users reacted to the video versus textual media, and the information gaps the explanations closed. The results also reveal issues of particular importance to explanations of this type.

Earlier research on gender effects with software features intended to help problem-solvers in end-user debugging environments has shown that females are less likely to use unfamiliar software features. This poses a serious problem because these features may be key to helping them with debugging problems. Contrasting this with research documenting males' inclination for tinkering in unfamiliar environments, the question arises as to whether encouraging tinkering with new features would help females overcome the factors, such as low self-efficacy, that led to the earlier results. In this paper, we present an experiment with males and females in an end-user debugging setting, and investigate how tinkering behavior impacts several measures of their debugging success. Our results show that the factors of tinkering, reflection, and self-efficacy, can combine in multiple ways to impact debugging effectiveness differently for males than for females.

We have been working to uncover gender differences in the ways males and females problem solve in end-user programming situations, and have discovered differences in males’ versus females’ use of several debugging features. Still, because this line of investigation is new, knowing exactly what to look for is difficult and important information could escape our notice. We therefore decided to bring data mining techniques to bear on our data, with two aims: primarily, to expand what is known about how males versus females make use of end-user debugging features, and secondarily, to find out whether data mining could bring new understanding to this research, given that we had already studied the data manually using qualitative and quantitative methods. The results suggested several new hypotheses in how males versus females go about end-user debugging tasks, the factors that play into their choices, and how their choices are associated with success.

The problem of dependability in end-user programming is an emerging area of interest. Pair collaboration in end-user software development may offer a way for end users to debug their programs more effectively. While pair programming studies - primarily of computer science students and professionals - report positive outcomes in terms of overall program quality, little is known about specific activities that pairs engage in that lead to those outcomes, or of how the previous results may pertain to end-user programmers. In this paper we analyze protocols of end-user pairs debugging spreadsheets. The results suggest that end-user pairs can achieve rich reasoning, effective planning, and systematic evaluation. Furthermore, end-user pairs provide specific types of mutual support that facilitate the accomplishment of their goals