Search

While there is widespread interest in supporting young people to critically evaluate machine learning-powered systems, there is little research on how we can support them in inquiring about how these systems work and what their limitations and implications may be. Outside of K-12 education, an effective strategy in evaluating black-boxed systems is algorithm auditing-a method for understanding algorithmic systems' opaque inner workings and external impacts from the outside in. In this paper, we review how expert researchers conduct algorithm audits and how end users engage in auditing practices to propose five steps that, when incorporated into learning activities, can support young people in auditing algorithms. We present a case study of a team of teenagers engaging with each step during an out-of-school workshop in which they audited peer-designed generative AI TikTok filters. We discuss the kind of scaffolds we provided to support youth in algorithm auditing and directions and challenges for integrating algorithm auditing into classroom activities. This paper contributes: (a) a conceptualization of five steps to scaffold algorithm auditing learning activities, and (b) examples of how youth engaged with each step during our pilot study.

Despite the growing accessibility of augmented reality (AR) for visualization, existing computer-aided design systems remain largely confined to traditional screens and are often inaccessible to novice users due to their complexity. We present RealityCraft, an open-sourced AR interface that enables in-situ computer-aided design and manufacturing (CAD+CAM) for novices. Unlike traditional CAD systems confined to computer screens, RealityCraft allows users to design directly within their physical environments, with primitive geometries. RealityCraft recognizes and utilizes physical constraints such as furniture and walls, enhancing user interaction through spatial awareness and depth occlusion. Furthermore, RealityCraft features an integrated AR-based 3D printing workflow, where users can drag and drop designs onto their 3D printer's virtual twin in their immediate space. Through a user study, we demonstrate that RealityCraft enhances engagement and ease of use for novices. By bridging the gap between digital creation and physical output, RealityCraft aims to transform everyday spaces into creative studios.

When designing an AI-assisted decision-making system, there is often a tradeoff between precision and recall in the AI's recommendations. We argue that careful exploitation of this tradeoff can harness the complementary strengths in the human-AI collaboration to significantly improve team performance. We investigate a real-world video anonymization task for which recall is paramount and more costly to improve. We analyze the performance of 78 professional annotators working with a) no AI assistance, b) a high-precision "restrained" AI, and c) a high-recall "zealous" AI in over 3,466 person-hours of annotation work. In comparison, the zealous AI helps human teammates achieve significantly shorter task completion time and higher recall. In a follow-up study, we remove AI assistance for everyone and find negative training effects on annotators trained with the restrained AI. These findings and our analysis point to important implications for the design of AI assistance in recall-demanding scenarios., Comment: 15 pages, 14 figures, accepted to ACM CHI 2023

Access to mental health support remains limited, particularly in marginalized communities where structural and cultural barriers hinder timely care. This paper explores the potential of AI-enabled chatbots as a scalable solution, focusing on advanced large language models (LLMs)-GPT v4, Mistral Large, and LLama V3.1-and assessing their ability to deliver empathetic, meaningful responses in mental health contexts. While these models show promise in generating structured responses, they fall short in replicating the emotional depth and adaptability of human therapists. Additionally, trustworthiness, bias, and privacy challenges persist due to unreliable datasets and limited collaboration with mental health professionals. To address these limitations, we propose a federated learning framework that ensures data privacy, reduces bias, and integrates continuous validation from clinicians to enhance response quality. This approach aims to develop a secure, evidence-based AI chatbot capable of offering trustworthy, empathetic, and bias-reduced mental health support, advancing AI's role in digital mental health care., Comment: 17 pages, 9 Figures

Mental health concerns are prevalent among college students, highlighting the need for effective interventions that promote self-awareness and holistic well-being. MindScape pioneers a novel approach to AI-powered journaling by integrating passively collected behavioral patterns such as conversational engagement, sleep, and location with Large Language Models (LLMs). This integration creates a highly personalized and context-aware journaling experience, enhancing self-awareness and well-being by embedding behavioral intelligence into AI. We present an 8-week exploratory study with 20 college students, demonstrating the MindScape app's efficacy in enhancing positive affect (7%), reducing negative affect (11%), loneliness (6%), and anxiety and depression, with a significant week-over-week decrease in PHQ-4 scores (-0.25 coefficient), alongside improvements in mindfulness (7%) and self-reflection (6%). The study highlights the advantages of contextual AI journaling, with participants particularly appreciating the tailored prompts and insights provided by the MindScape app. Our analysis also includes a comparison of responses to AI-driven contextual versus generic prompts, participant feedback insights, and proposed strategies for leveraging contextual AI journaling to improve well-being on college campuses. By showcasing the potential of contextual AI journaling to support mental health, we provide a foundation for further investigation into the effects of contextual AI journaling on mental health and well-being., Comment: arXiv admin note: text overlap with arXiv:2404.00487

Studies on human decision-making focused on humanitarian aid have found that cognitive biases can hinder the fair allocation of resources. However, few HCI and Information Visualization studies have explored ways to overcome those cognitive biases. This work investigates whether the design of interactive resource allocation tools can help to promote allocation fairness. We specifically study the effect of presentation format (using text or visualization) and a specific framing strategy (showing resources allocated to groups or individuals). In our three crowdsourced experiments, we provided different tool designs to split money between two fictional programs that benefit two distinct communities. Our main finding indicates that individual-framed visualizations and text may be able to curb unfair allocations caused by group-framed designs. This work opens new perspectives that can motivate research on how interactive tools and visualizations can be engineered to combat cognitive biases that lead to inequitable decisions., Comment: Published as a conference paper at CHI 2023

Older adults are increasingly acquiring smartphones. But acquiring smartphones can be difficult, and little is known about the particular challenges of older adults who are additionally blind or losing their vision. We shed light on the social and technical aspects of acquiring smartphones with vision loss, based on deep qualitative interviews with 22 blind or low vision (BLV) older adults aged 60 and over. Through our grounded theory analysis, we found that BLV older adults experience liminality as they acquire smartphones and transition through re-acquiring smartphones as they become blind, and they can transition through liminality by participating in mutual aid within the blind community. We contribute the notion of "Intersecting Liminality," which explains the marginalizing experience of simultaneously transitioning through vision loss, aging, and technology acquisition. We contend that Intersecting Liminality can serve as a framework that centers the dynamic nature of disability to help our community generate a more nuanced understanding of technology acquisition and more effective assistive interventions., Comment: 14 pages, 2 figures, 2 tables, conference paper accepted to The 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2024)

Complexity is often seen as a inherent negative in information design, with the job of the designer being to reduce or eliminate complexity, and with principles like Tufte's "data-ink ratio" or "chartjunk" to operationalize minimalism and simplicity in visualizations. However, in this position paper, we call for a more expansive view of complexity as a design material, like color or texture or shape: an element of information design that can be used in many ways, many of which are beneficial to the goals of using data to understand the world around us. We describe complexity as a phenomenon that occurs not just in visual design but in every aspect of the sensemaking process, from data collection to interpretation. For each of these stages, we present examples of ways that these various forms of complexity can be used (or abused) in visualization design. We ultimately call on the visualization community to build a more nuanced view of complexity, to look for places to usefully integrate complexity in multiple stages of the design process, and, even when the goal is to reduce complexity, to look for the non-visual forms of complexity that may have otherwise been overlooked., Comment: Manuscript accepted for BELIV 2024, 10 pages, 3 figures

Understanding and communicating data uncertainty is crucial for informed decision-making across various domains, including finance, healthcare, and public policy. This study investigates the impact of gender and acoustic variables on decision-making, confidence, and trust through a crowdsourced experiment. We compared visualization-only representations of uncertainty to text-forward and speech-forward bimodal representations, including multiple synthetic voices across gender. Speech-forward representations led to an increase in risky decisions, and text-forward representations led to lower confidence. Contrary to prior work, speech-forward forecasts did not receive higher ratings of trust. Higher normalized pitch led to a slight increase in decision confidence, but other voice characteristics had minimal impact on decisions and trust. An exploratory analysis of accented speech showed consistent results with the main experiment and additionally indicated lower trust ratings for information presented in Indian and Kenyan accents. The results underscore the importance of considering acoustic and contextual factors in presentation of data uncertainty., Comment: 11 pages, 5 figures, Uncertainty Workshop at IEEE VIS 2024 Conference

Written language is a useful tool for non-visual creative activities like writing essays and planning searches. This paper investigates the integration of written language in to the visualization design process. We create the idea of a 'writing rudder,' which acts as a guiding force or strategy for the design. Via an interview study of 24 working visualization designers, we first established that only a minority of participants systematically use writing to aid in design. A second study with 15 visualization designers examined four different variants of written rudders: asking questions, stating conclusions, composing a narrative, and writing titles. Overall, participants had a positive reaction; designers recognized the benefits of explicitly writing down components of the design and indicated that they would use this approach in future design work. More specifically, two approaches - writing questions and writing conclusions/takeaways - were seen as beneficial across the design process, while writing narratives showed promise mainly for the creation stage. Although concerns around potential bias during data exploration were raised, participants also discussed strategies to mitigate such concerns. This paper contributes to a deeper understanding of the interplay between language and visualization, and proposes a straightforward, lightweight addition to the visualization design process., Comment: 11 pages, 3 figures, accepted for IEEE VIS conference 2024

Traditional recommendation proposals, including content-based and collaborative filtering, usually focus on similarity between items or users. Existing approaches lack ways of introducing unexpectedness into recommendations, prioritizing globally popular items over exposing users to unforeseen items. This investigation aims to design and evaluate a novel layer on top of recommender systems suited to incorporate relational information and suggest items with a user-defined degree of surprise. We propose a Knowledge Graph (KG) based recommender system by encoding user interactions on item catalogs. Our study explores whether network-level metrics on KGs can influence the degree of surprise in recommendations. We hypothesize that surprisingness correlates with certain network metrics, treating user profiles as subgraphs within a larger catalog KG. The achieved solution reranks recommendations based on their impact on structural graph metrics. Our research contributes to optimizing recommendations to reflect the metrics. We experimentally evaluate our approach on two datasets of LastFM listening histories and synthetic Netflix viewing profiles. We find that reranking items based on complex network metrics leads to a more unexpected and surprising composition of recommendation lists.

Behavioral sensing technologies are rapidly evolving across a range of well-being applications. Despite its potential, concerns about the responsible use of such technology are escalating. In response, recent research within the sensing technology has started to address these issues. While promising, they primarily focus on broad demographic categories and overlook more nuanced, context-specific identities. These approaches lack grounding within domain-specific harms that arise from deploying sensing technology in diverse social, environmental, and technological settings. Additionally, existing frameworks for evaluating harms are designed for a generic ML life cycle, and fail to adapt to the dynamic and longitudinal considerations for behavioral sensing technology. To address these gaps, we introduce a framework specifically designed for evaluating behavioral sensing technologies. This framework emphasizes a comprehensive understanding of context, particularly the situated identities of users and the deployment settings of the sensing technology. It also highlights the necessity for iterative harm mitigation and continuous maintenance to adapt to the evolving nature of technology and its use. We demonstrate the feasibility and generalizability of our framework through post-hoc evaluations on two real-world behavioral sensing studies conducted in different international contexts, involving varied population demographics and machine learning tasks. Our evaluations provide empirical evidence of both situated identity-based harm and more domain-specific harms, and discuss the trade-offs introduced by implementing bias mitigation techniques., Comment: 26 pages, 8 tables, and 1 figure (excluding appendix)

Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Artificial Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the ability to interact as if they were digital, where every object has the potential to serve as a portal to digital functionalities. Our approach utilizes real-time object segmentation and classification, combined with the power of Multimodal Large Language Models (MLLMs), to facilitate these interactions without the need for object pre-registration. We implement the AOI concept in the form of XR-Objects, an open-source prototype system that provides a platform for users to engage with their physical environment in contextually relevant ways using object-based context menus. This system enables analog objects to not only convey information but also to initiate digital actions, such as querying for details or executing tasks. Our contributions are threefold: (1) we define the AOI concept and detail its advantages over traditional AI assistants, (2) detail the XR-Objects system's open-source design and implementation, and (3) show its versatility through various use cases and a user study., Comment: To appear in the Proceedings of the 2024 ACM Symposium on User Interface Software and Technology (UIST)

MindScape aims to study the benefits of integrating time series behavioral patterns (e.g., conversational engagement, sleep, location) with Large Language Models (LLMs) to create a new form of contextual AI journaling, promoting self-reflection and well-being. We argue that integrating behavioral sensing in LLMs will likely lead to a new frontier in AI. In this Late-Breaking Work paper, we discuss the MindScape contextual journal App design that uses LLMs and behavioral sensing to generate contextual and personalized journaling prompts crafted to encourage self-reflection and emotional development. We also discuss the MindScape study of college students based on a preliminary user study and our upcoming study to assess the effectiveness of contextual AI journaling in promoting better well-being on college campuses. MindScape represents a new application class that embeds behavioral intelligence in AI.

MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: \textit{``I have felt down, depressed, or hopeless''}. Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.

University students encounter various forms of stress during their academic journey, including cyclical stress associated with final exams. Supporting their well-being means helping them manage their stress levels. In this study, we used a wearable health-tracking ring on a cohort of 103 Japanese university students for up to 28 months in the wild. The study aimed to investigate whether group-wide biomarkers of stress can be identified in a sample having similar daily schedules and whether these occurrences can be pinpointed to specific periods of the academic year. We found population-wide increased stress markers during exams, New Year's, and job hunting season, a Japanese job market peculiarity. Our results highlight the available potential of unobtrusive, in-situ detection of the current mental state of university student populations using off-the-shelf wearables from noisy data, with significant implications for the well-being of the users. Our approach and method of analysis allows for monitoring the student body's stress level without singling out individuals and therefore represents a privacy-preserving method. This way, new and sudden stress increases can be recognized, which can help identify the stressor and inform the design and introduction of counter measures., Comment: 19 pages, 6 figures, 5 tables

This paper addresses the complexities inherent in AI product prototyping, focusing on the challenges posed by the probabilistic nature of AI behavior and the limited accessibility of prototyping tools to non-experts. A Design Science Research (DSR) approach is presented which culminates in a conceptual framework aimed at improving the AI prototyping process. Through a comprehensive literature review, key challenges were identified and no-code AutoML was analyzed as a solution. The framework describes the seamless incorporation of non-expert input and evaluation during prototyping, leveraging the potential of no-code AutoML to enhance accessibility and interpretability. A hybrid approach of combining naturalistic (case study) and artificial evaluation methods (criteria-based analysis) validated the utility of our approach, highlighting its efficacy in supporting AI non-experts and streamlining decision-making and its limitations. Implications for academia and industry, emphasizing the strategic integration of no-code AutoML to enhance AI product development processes, mitigate risks, and foster innovation, are discussed.

This paper investigates the role of text in visualizations, specifically the impact of text position, semantic content, and biased wording. Two empirical studies were conducted based on two tasks (predicting data trends and appraising bias) using two visualization types (bar and line charts). While the addition of text had a minimal effect on how people perceive data trends, there was a significant impact on how biased they perceive the authors to be. This finding revealed a relationship between the degree of bias in textual information and the perception of the authors' bias. Exploratory analyses support an interaction between a person's prediction and the degree of bias they perceived. This paper also develops a crowdsourced method for creating chart annotations that range from neutral to highly biased. This research highlights the need for designers to mitigate potential polarization of readers' opinions based on how authors' ideas are expressed., Comment: 12 pages, 7 figures, for supplemental materials: https://github.com/chasejstokes/role-text

Unraveling the role of opinion leaders in the digital realm, this study investigates the influence of key actors on Telegram, a hybrid platform that combines messaging app features with social network dynamics, where channel administrators gain a unique authoritative role. This research aims to create a method to identify opinion leaders in a network of forwarded messages on Telegram, adapting a method originally developed to be applied to Twitter. The adapted method is showcased through a case study during the 2022 Brazilian Presidential Election, involving the monitoring of 25 pro-Bolsonaro groups. The findings contribute to understanding the dynamics of digital opinion leadership, particularly in politically charged environments., Comment: 7 pages, 3 figures, submitted to ACM Web Sci 2024

Social isolation is a common problem faced by individuals with serious mental illness (SMI), and current intervention approaches have limited effectiveness. This paper presents a blended intervention approach, called mobile Social Interaction Therapy by Exposure (mSITE), to address social isolation in individuals with serious mental illness. The approach combines brief in-person cognitive-behavioral therapy (CBT) with context-triggered mobile CBT interventions that are personalized using mobile sensing data. Our approach targets social behavior and is the first context-aware intervention for improving social outcomes in serious mental illness.

Machine learning technology has become ubiquitous, but, unfortunately, often exhibits bias. As a consequence, disparate stakeholders need to interact with and make informed decisions about using machine learning models in everyday systems. Visualization technology can support stakeholders in understanding and evaluating trade-offs between, for example, accuracy and fairness of models. This paper aims to empirically answer "Can visualization design choices affect a stakeholder's perception of model bias, trust in a model, and willingness to adopt a model?" Through a series of controlled, crowd-sourced experiments with more than 1,500 participants, we identify a set of strategies people follow in deciding which models to trust. Our results show that men and women prioritize fairness and performance differently and that visual design choices significantly affect that prioritization. For example, women trust fairer models more often than men do, participants value fairness more when it is explained using text than as a bar chart, and being explicitly told a model is biased has a bigger impact than showing past biased performance. We test the generalizability of our results by comparing the effect of multiple textual and visual design choices and offer potential explanations of the cognitive mechanisms behind the difference in fairness perception and trust. Our research guides design considerations to support future work developing visualization systems for machine learning., Comment: 11 pages, 6 figures, to appear in IEEE Transactions of Visualization and Computer Graphics (Also in proceedings of IEEE VIS 2023)

Since late 2022, Large Language Models (LLMs) have become very prominent with LLMs like ChatGPT and Bard receiving millions of users. Hundreds of new LLMs are announced each week, many of which are deposited to Hugging Face, a repository of machine learning models and datasets. To date, nearly 16,000 Text Generation models have been uploaded to the site. Given the huge influx of LLMs, it is of interest to know which LLM backbones, settings, training methods, and families are popular or trending. However, there is no comprehensive index of LLMs available. We take advantage of the relatively systematic nomenclature of Hugging Face LLMs to perform hierarchical clustering and identify communities amongst LLMs using n-grams and term frequency-inverse document frequency. Our methods successfully identify families of LLMs and accurately cluster LLMs into meaningful subgroups. We present a public web application to navigate and explore Constellation, our atlas of 15,821 LLMs. Constellation rapidly generates a variety of visualizations, namely dendrograms, graphs, word clouds, and scatter plots. Constellation is available at the following link: https://constellation.sites.stanford.edu/., Comment: 14 pages, 6 figures, 1 table

New systems employ Machine Learning to sift through large knowledge sources, creating flexible Large Language Models. These models discern context and predict sequential information in various communication forms. Generative AI, leveraging Transformers, generates textual or visual outputs mimicking human responses. It proposes one or multiple contextually feasible solutions for a user to contemplate. However, generative AI does not currently support traceability of ideas, a useful feature provided by search engines indicating origin of information. The narrative style of generative AI has gained positive reception. People learn from stories. Yet, early ChatGPT efforts had difficulty with truth, reference, calculations, and aspects like accurate maps. Current capabilities of referencing locations and linking to apps seem to be better catered by the link-centric search methods we've used for two decades. Deploying truly believable solutions extends beyond simulating contextual relevance as done by generative AI. Combining the creativity of generative AI with the provenance of internet sources in hybrid scenarios could enhance internet usage. Generative AI, viewed as drafts, stimulates thinking, offering alternative ideas for final versions or actions. Scenarios for information requests are considered. We discuss how generative AI can boost idea generation by eliminating human bias. We also describe how search can verify facts, logic, and context. The user evaluates these generated ideas for selection and usage. This paper introduces a system for knowledge workers, Generate And Search Test, enabling individuals to efficiently create solutions previously requiring top collaborations of experts., Comment: 8 pages, 21 references

Order is one of the main instruments to measure the relationship between objects in (empirical) data. However, compared to methods that use numerical properties of objects, the amount of ordinal methods developed is rather small. One reason for this is the limited availability of computational resources in the last century that would have been required for ordinal computations. Another reason -- particularly important for this line of research -- is that order-based methods are often seen as too mathematically rigorous for applying them to real-world data. In this paper, we will therefore discuss different means for measuring and 'calculating' with ordinal structures -- a specific class of directed graphs -- and show how to infer knowledge from them. Our aim is to establish Ordinal Data Science as a fundamentally new research agenda. Besides cross-fertilization with other cornerstone machine learning and knowledge representation methods, a broad range of disciplines will benefit from this endeavor, including, psychology, sociology, economics, web science, knowledge engineering, scientometrics., Comment: 40 pages, 7 figures, Transactions on Graph Data and Knowledge (TGDK)

Multiple-view visualizations (MVs) have been widely used for visual analysis. Each view shows some part of the data in a usable way, and together multiple views enable a holistic understanding of the data under investigation. For example, an analyst may check a social network graph, a map of sensitive locations, a table of transaction records, and a collection of reports to identify suspicious activities. While each view is designed to preserve its own visual context with visible borders or perceivable spatial distance from others, the key to solving real-world analysis problems often requires "breaking" such boundaries, and further integrating and synthesizing the data scattered across multiple views. This calls for blending the boundaries in MVs, instead of simply breaking them, which brings key questions: what are possible boundaries in MVs, and what are design options that can support the boundary blending in MVs? To answer these questions, we present three boundaries in MVs: 1) data boundary, 2) representation boundary, and 3) semantic boundary, corresponding to three major aspects regarding the usage of MVs: encoded information, visual representation, and interpretation. Then, we discuss four design strategies (highlighting, linking, embedding, and extending) and their pros and cons for supporting boundary blending in MVs. We conclude our discussion with future research opportunities.

Online whiteboards are becoming a popular way to facilitate collaborative design work, providing a free-form environment to curate ideas. However, as templates are increasingly being used to scaffold contributions from non-experts designers, it is crucial to understand their impact on the creative process. In this paper, we present the results from a study with 114 students in a large introductory design course. Our results confirm prior findings that templates benefit students by providing a starting point, a shared process, and the ability to access their own work from previous steps. While prior research has criticized templates for being too rigid, we discovered that using templates within a free-form environment resulted in visual patterns of free-form curation where concepts were spatially organized, clustered, color-coded, and connected using arrows and lines. We introduce the concept of "Free-form Templates" to illustrate how templates and free-form curation can be synergistic.

The security and privacy of refugee communities have emerged as pressing concerns in the context of increasing global migration. The Rohingya refugees are a stateless Muslim minority group in Myanmar who were forced to flee their homes after conflict broke out, with many fleeing to neighbouring countries and ending up in refugee camps, such as in Bangladesh. However, others migrated to Malaysia and those who arrive there live within the community as urban refugees. However, the Rohingya in Malaysia are not legally recognized and have limited and restricted access to public resources such as healthcare and education. This means they face security and privacy challenges, different to other refugee groups, which are often compounded by this lack of recognition, social isolation and lack of access to vital resources. This paper discusses the implications of security and privacy of the Rohingya refugees, focusing on available and accessible technological assistance, uncovering the heightened need for a human-centered approach to design and implementation of solutions that factor in these requirements. Overall, the discussions and findings presented in this paper on the security and privacy of the Rohingya provides a valuable resource for researchers, practitioners and policymakers in the wider HCI community., Comment: 5 pages, 3 figures, CHI'23 Workshop on Migration, Security and Privacy (see https://migrationsecurityprivacy.uk)

Trust is a key motivation in developing explainable artificial intelligence (XAI). However, researchers attempting to measure trust in AI face numerous challenges, such as different trust conceptualizations, simplified experimental tasks that may not induce uncertainty as a prerequisite for trust, and the lack of validated trust questionnaires in the context of AI. While acknowledging these issues, we have identified a further challenge that currently seems underappreciated - the potential distinction between trust as one construct and \emph{distrust} as a second construct independent of trust. While there has been long-standing academic discourse for this distinction and arguments for both the one-dimensional and two-dimensional conceptualization of trust, distrust seems relatively understudied in XAI. In this position paper, we not only highlight the theoretical arguments for distrust as a distinct construct from trust but also contextualize psychometric evidence that likewise favors a distinction between trust and distrust. It remains to be investigated whether the available psychometric evidence is sufficient for the existence of distrust or whether distrust is merely a measurement artifact. Nevertheless, the XAI community should remain receptive to considering trust and distrust for a more comprehensive understanding of these two relevant constructs in XAI.

Intelligent writing assistants powered by large language models (LLMs) are more popular today than ever before, but their further widespread adoption is precluded by sub-optimal performance. In this position paper, we argue that a major reason for this sub-optimal performance and adoption is a singular focus on the information content of language while ignoring its social aspects. We analyze the different dimensions of these social factors in the context of writing assistants and propose their incorporation into building smarter, more effective, and truly personalized writing assistants that would enrich the user experience and contribute to increased user adoption., Comment: 2 pages, Accepted to In2Writing Workshop (CHI 2023)

Improving object manipulation skills through hand-object interaction exercises is crucial for rehabilitation. Despite limited healthcare resources, physical therapists propose remote exercise routines followed up by remote monitoring. However, remote motor skills assessment remains challenging due to the lack of effective motion visualizations. Therefore, exploring innovative ways of visualization is crucial, and virtual reality (VR) has shown the potential to address this limitation. However, it is unclear how VR visualization can represent understandable hand-object interactions. To address this gap, in this paper, we present VRMoVi, a VR visualization system that incorporates multiple levels of 3D visualization layers to depict movements. In a 2-stage study, we showed VRMoVi's potential in representing hand-object interactions, with its visualization outperforming traditional representations, and detailed features improved the hand-object interactions understanding. This study takes the initial step in developing VR visualization of hand-object interaction to support remote physical therapy., Comment: 12 pages, 3 figures

Deepfakes are AI-synthesized content that are becoming popular on many social media platforms, meaning the use of deepfakes is increasing in society, regardless of its societal implications. Its implications are harmful if the moral intuitions behind deepfakes are problematic; thus, it is important to explore how the moral intuitions behind deepfakes unfold in communities at scale. However, understanding perceived moral viewpoints unfolding in digital contexts is challenging, due to the complexities in conversations. In this research, we demonstrate how Moral Foundations Theory (MFT) can be used as a lens through which to operationalize moral viewpoints in discussions about deepfakes on Reddit communities. Using the extended Moral Foundations Dictionary (eMFD), we measured the strengths of moral intuition (moral loading) behind 101,869 Reddit posts. We present the discussions that unfolded on Reddit in 2018 to 2022 wherein intuitions behind some posts were found to be morally questionable to society. Our results may help platforms detect and take action against immoral activities related to deepfakes., Comment: This is more related to CSCW, societal implications. Manuscript to be submitted to SM+S Journal (Sage Group)

Graphical perception studies typically measure visualization encoding effectiveness using the error of an "average observer", leading to canonical rankings of encodings for numerical attributes: e.g., position > area > angle > volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this ranking across individuals not captured by population-level metrics using "average observer" models. One way we can bridge this gap is by recasting classic visual perception tasks as tools for assessing individual performance, in addition to overall visualization performance. In this paper we replicate and extend Cleveland and McGill's graphical comparison experiment using Bayesian multilevel regression, using these models to explore individual differences in visualization skill from multiple perspectives. The results from experiments and modeling indicate that some people show patterns of accuracy that credibly deviate from the canonical rankings of visualization effectiveness. We discuss implications of these findings, such as a need for new ways to communicate visualization effectiveness to designers, how patterns in individuals' responses may show systematic biases and strategies in visualization judgment, and how recasting classic visual perception tasks as tools for assessing individual performance may offer new ways to quantify aspects of visualization literacy. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/8ub7t/?view\_only=9be4798797404a4397be3c6fc2a68cc0., Comment: 16 pages, 9 figures

Gardening is an activity that involves a number of dimensions of increasing interest to HCI and CSCW researchers, including recreation, sustainability, and engagement with nature. This paper considers the garden setting in order to understand the role that collaborative and social computing technologies might play for practitioners engaging in outdoor skilled activities. We conducted participant observations with nine experienced gardeners aged 22-71 years. Through this process, we find that gardeners continuously configure their environments to accommodate their preferences for sociality. They share embodied skills and help others attune to sensory information in person, but also influence learning through the features in their garden that are observed by others. This paper provides an understanding of sociality in the garden, highlights skill sharing as a key domain for design in this space, and contributes design considerations for collaborative technologies in outdoor settings., Comment: 13 pages

Physical therapy (PT) is crucial for patients to restore and maintain mobility, function, and well-being. Many on-site activities and body exercises are performed under the supervision of therapists or clinicians. However, the postures of some exercises at home cannot be performed accurately due to the lack of supervision, quality assessment, and self-correction. Therefore, in this paper, we design a new framework, PhysiQ, that continuously tracks and quantitatively measures people's off-site exercise activity through passive sensory detection. In the framework, we create a novel multi-task spatio-temporal Siamese Neural Network that measures the absolute quality through classification and relative quality based on an individual's PT progress through similarity comparison. PhysiQ digitizes and evaluates exercises in three different metrics: range of motions, stability, and repetition., Comment: 22 pages

Given a choice between charts with minimal text and those with copious textual annotations, participants in a study (Stokes et al.) tended to prefer the charts with more text. This paper examines the qualitative responses of the participants' preferences for various stimuli integrating charts and text, including a text-only variant. A thematic analysis of these responses resulted in three main findings. First, readers commented most frequently on the presence or lack of context; they preferred to be informed, even when it sacrificed simplicity. Second, readers discussed the story-like component of the text-only variant and made little mention of narrative in relation to the chart variants. Finally, readers showed suspicion around possible misleading elements of the chart or text. These themes support findings from previous work on annotations, captions, and alternative text. We raise further questions regarding the combination of text and visual communication., Comment: 7 pages, 3 figures, accepted to the NLVIZ workshop at IEEE Transaction on Visualization and Graphics conference

Dynamic networks reflect temporal changes occurring to the graph's structure and are used to model a wide variety of problems in many application fields. We investigate the design space of dynamic graph visualization along two major dimensions: the network structural and temporal representation. Significant research has been conducted evaluating the benefits and drawbacks of different structural representations for static graphs, however, few extend this comparison to a dynamic network setting. We conduct a study where we assess the participants' response times, accuracy, and preferences for different combinations of the graph's structural and temporal representations on typical dynamic network exploration tasks, with and without support of common interaction methods. Our results suggest that matrices provide better support for tasks on lower-level entities and basic interactions require longer response times while increasing accuracy. Node-link with auto animation proved to be the quickest and most accurate combination overall, while animation with playback control the most preferred temporal encoding., Comment: Appears in the Proceedings of the 30th International Symposium on Graph Drawing and Network Visualization (GD 2022)

The language for expressing comparisons is often complex and nuanced, making supporting natural language-based visual comparison a non-trivial task. To better understand how people reason about comparisons in natural language, we explore a design space of utterances for comparing data entities. We identified different parameters of comparison utterances that indicate what is being compared (i.e., data variables and attributes) as well as how these parameters are specified (i.e., explicitly or implicitly). We conducted a user study with sixteen data visualization experts and non-experts to investigate how they designed visualizations for comparisons in our design space. Based on the rich set of visualization techniques observed, we extracted key design features from the visualizations and synthesized them into a subset of sixteen representative visualization designs. We then conducted a follow-up study to validate user preferences for the sixteen representative visualizations corresponding to utterances in our design space. Findings from these studies suggest guidelines and future directions for designing natural language interfaces and recommendation tools to better support natural language comparisons in visual analytics., Comment: 11 pages, 5 figures, 1 table, to appear in IEEE Transactions of Visualization and Computer Graphics (Also in proceedings of IEEE VIS 2022)

Despite the ever-growing popularity of dashboards across a wide range of domains, their authoring still remains a tedious and complex process. Current tools offer considerable support for creating individual visualizations but provide limited support for discovering groups of visualizations that can be collectively useful for composing analytic dashboards. To address this problem, we present MEDLEY, a mixed-initiative interface that assists in dashboard composition by recommending dashboard collections (i.e., a logically grouped set of views and filtering widgets) that map to specific analytical intents. Users can specify dashboard intents (namely, measure analysis, change analysis, category analysis, or distribution analysis) explicitly through an input panel in the interface or implicitly by selecting data attributes and views of interest. The system recommends collections based on these analytic intents, and views and widgets can be selected to compose a variety of dashboards. MEDLEY also provides a lightweight direct manipulation interface to configure interactions between views in a dashboard. Based on a study with 13 participants performing both targeted and open-ended tasks, we discuss how MEDLEY's recommendations guide dashboard composition and facilitate different user workflows. Observations from the study identify potential directions for future work, including combining manual view specification with dashboard recommendations and designing natural language interfaces for dashboard authoring., Comment: 11 pages, 8 figures, 2 tables, accepted to IEEE Transaction on Visualization and Graphics (also appearing in the proceedings of IEEE VIS 2022)

While visualizations are an effective way to represent insights about information, they rarely stand alone. When designing a visualization, text is often added to provide additional context and guidance for the reader. However, there is little experimental evidence to guide designers as to what is the right amount of text to show within a chart, what its qualitative properties should be, and where it should be placed. Prior work also shows variation in personal preferences for charts versus textual representations. In this paper, we explore several research questions about the relative value of textual components of visualizations. 302 participants ranked univariate line charts containing varying amounts of text, ranging from no text (except for the axes) to a written paragraph with no visuals. Participants also described what information they could take away from line charts containing text with varying semantic content. We find that heavily annotated charts were not penalized. In fact, participants preferred the charts with the largest number of textual annotations over charts with fewer annotations or text alone. We also find effects of semantic content. For instance, the text that describes statistical or relational components of a chart leads to more takeaways referring to statistics or relational comparisons than text describing elemental or encoded components. Finally, we find different effects for the semantic levels based on the placement of the text on the chart; some kinds of information are best placed in the title, while others should be placed closer to the data. We compile these results into four chart design guidelines and discuss future implications for the combination of text and charts., Comment: 11 pages, 4 tables, 6 figures, accepted to IEEE Transaction on Visualization and Graphics

Trust is often cited as an essential criterion for the effective use and real-world deployment of AI. Researchers argue that AI should be more transparent to increase trust, making transparency one of the main goals of XAI. Nevertheless, empirical research on this topic is inconclusive regarding the effect of transparency on trust. An explanation for this ambiguity could be that trust is operationalized differently within XAI. In this position paper, we advocate for a clear distinction between behavioral (objective) measures of reliance and attitudinal (subjective) measures of trust. However, researchers sometimes appear to use behavioral measures when intending to capture trust, although attitudinal measures would be more appropriate. Based on past research, we emphasize that there are sound theoretical reasons to keep trust and reliance separate. Properly distinguishing these two concepts provides a more comprehensive understanding of how transparency affects trust and reliance, benefiting future XAI research., Comment: Will be published at the CHI 2022 Workshop on Trust and Reliance in AI-Human Teams. New Orleans '22: ACM Conference on Human Factors in Computing Systems

Existing approaches for embedding unobtrusive tags inside 3D objects require either complex fabrication or high-cost imaging equipment. We present InfraredTags, which are 2D markers and barcodes imperceptible to the naked eye that can be 3D printed as part of objects, and detected rapidly by low-cost near-infrared cameras. We achieve this by printing objects from an infrared-transmitting filament, which infrared cameras can see through, and by having air gaps inside for the tag's bits, which appear at a different intensity in the infrared image. We built a user interface that facilitates the integration of common tags (QR codes, ArUco markers) with the object geometry to make them 3D printable as InfraredTags. We also developed a low-cost infrared imaging module that augments existing mobile devices and decodes tags using our image processing pipeline. Our evaluation shows that the tags can be detected with little near-infrared illumination (0.2lux) and from distances as far as 250cm. We demonstrate how our method enables various applications, such as object tracking and embedding metadata for augmented reality and tangible interactions., Comment: 12 pages, 10 figures. To appear in the Proceedings of the 2022 ACM Conference on Human Factors in Computing Systems

As emerging technologies increasingly integrate into all facets of our lives, the workplace stands at the forefront of potential transformative changes. A notable development in this realm is the advent of passive sensing technology, designed to enhance both cognitive and physical capabilities by monitoring human behavior. This paper reviews current research on the application of passive sensing technology in the workplace, focusing on its impact on employee wellbeing and productivity. Additionally, we explore unresolved issues and outline prospective pathways for the incorporation of passive sensing in future workplaces., Comment: Added references and other minor revisions. Also udated to include relevant works published after 2022

Probabilistic graphs are challenging to visualize using the traditional node-link diagram. Encoding edge probability using visual variables like width or fuzziness makes it difficult for users of static network visualizations to estimate network statistics like densities, isolates, path lengths, or clustering under uncertainty. We introduce Network Hypothetical Outcome Plots (NetHOPs), a visualization technique that animates a sequence of network realizations sampled from a network distribution defined by probabilistic edges. NetHOPs employ an aggregation and anchoring algorithm used in dynamic and longitudinal graph drawing to parameterize layout stability for uncertainty estimation. We present a community matching algorithm to enable visualizing the uncertainty of cluster membership and community occurrence. We describe the results of a study in which 51 network experts used NetHOPs to complete a set of common visual analysis tasks and reported how they perceived network structures and properties subject to uncertainty. Participants' estimates fell, on average, within 11% of the ground truth statistics, suggesting NetHOPs can be a reasonable approach for enabling network analysts to reason about multiple properties under uncertainty. Participants appeared to articulate the distribution of network statistics slightly more accurately when they could manipulate the layout anchoring and the animation speed. Based on these findings, we synthesize design recommendations for developing and using animated visualizations for probabilistic networks., Comment: 9 pages, 10 figures, 1 table. Accepted by IEEE VIS 2021

Data-driven decision making has been a common task in today's big data era, from simple choices such as finding a fast way to drive home, to complex decisions on medical treatment. It is often supported by visual analytics. For various reasons (e.g., system failure, interrupted network, intentional information hiding, or bias), visual analytics for sensemaking of data involves missingness (e.g., data loss and incomplete analysis), which impacts human decisions. For example, missing data can cost a business millions of dollars, and failing to recognize key evidence can put an innocent person in jail. Being aware of missingness is critical to avoid such catastrophes. To fulfill this, as an initial step, we consider missingness in visual analytics from two aspects: data-centric and human-centric. The former emphasizes missingness in three data-related categories: data composition, data relationship, and data usage. The latter focuses on the human-perceived missingness at three levels: observed-level, inferred-level, and ignored-level. Based on them, we discuss possible roles of visualizations for handling missingness, and conclude our discussion with future research opportunities., Comment: IEEE VIS (InfoVis/VAST/SciVis) 2022

We present how we have created a series of bilingual (English and Welsh) STEM activities focusing on rainbows, colours, light and optical effects. The activities were motivated by the many rainbows that appeared in windows in the UK, in support of the National Health Service at the start of the coronavirus pandemic. Rainbows are hopeful and are very fitting to be used as a positive iconic image at a time of much uncertainty. In this paper we explain how we have developed and organised the activities, focusing on colours, computer graphics and computer programming. Each lesson contains one or more activities, which enable people to take an active role in their learning. We have carefully prepared and organised several processes to guide academic colleagues to create and publish different activities in the theme. Which means that the activities appear similarly structured, can be categorised and searched in a consistent way. This structure can act as a blueprint for others to follow and apply to develop their own online course. The activities incrementally take people through learning about colour, rainbows, planning what to program, design and strategies to create colourful pictures using simple computer graphics principles based in processing.org., Comment: 9 pages, 10 figures, 1 table. Accepted for publication in Proceedings EG UK Computer Graphics & Visual Computing (2021)

Problem-driven visualization work is rooted in deeply understanding the data, actors, processes, and workflows of a target domain. However, an individual's personality traits and cognitive abilities may also influence visualization use. Diverse user needs and abilities raise natural questions for specificity in visualization design: Could individuals from different domains exhibit performance differences when using visualizations? Are any systematic variations related to their cognitive abilities? This study bridges domain-specific perspectives on visualization design with those provided by cognition and perception. We measure variations in visualization task performance across chemistry, computer science, and education, and relate these differences to variations in spatial ability. We conducted an online study with over 60 domain experts consisting of tasks related to pie charts, isocontour plots, and 3D scatterplots, and grounded by a well-documented spatial ability test. Task performance (correctness) varied with profession across more complex visualizations, but not pie charts, a comparatively common visualization. We found that correctness correlates with spatial ability, and the professions differ in terms of spatial ability. These results indicate that domains differ not only in the specifics of their data and tasks, but also in terms of how effectively their constituent members engage with visualizations and their cognitive traits. Analyzing participants' confidence and strategy comments suggests that focusing on performance neglects important nuances, such as differing approaches to engage with even common visualizations and potential skill transference. Our findings offer a fresh perspective on discipline-specific visualization with recommendations to help guide visualization design that celebrates the uniqueness of the disciplines and individuals we seek to serve., Comment: The paper has been accepted to IEEE VIS 2021, and will appear in IEEE TVCG. 11 pages with 9 figures

Whether investigating research questions or designing systems, many researchers and designers need to engage users with their personal data. However, it is difficult to successfully design user-facing tools for interacting with personal data without first understanding what users want to do with their data. Techniques for raw data exploration, sketching, or physicalization can avoid the perils of tool development, but prevent direct analytical access to users' rich personal data. We present a new method that directly tackles this challenge: the data engagement interview. This interview method incorporates an analyst to provide real-time personal data analysis, granting interview participants the opportunity to directly engage with their data, and interviewers to observe and ask questions throughout this engagement. We describe the method's development through a case study with asthmatic participants, share insights and guidance from our experience, and report a broad set of insights from these interviews., Comment: 22 pages, 3 figures, submitted to The Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) December 2021

A key task in design work is grasping the client's implicit tastes. Designers often do this based on a set of examples from the client. However, recognizing a common pattern among many intertwining variables such as color, texture, and layout and synthesizing them into a composite preference can be challenging. In this paper, we leverage the pattern recognition capability of computational models to aid in this task. We offer a set of principles for computationally learning personal style. The principles are manifested in PseudoClient, a deep learning framework that learns a computational model for personal graphic design style from only a handful of examples. In several experiments, we found that PseudoClient achieves a 79.40% accuracy with only five positive and negative examples, outperforming several alternative methods. Finally, we discuss how PseudoClient can be utilized as a building block to support the development of future design applications.

This article addresses the possibility of supporting the construction of emotions in the participants of Interactive Digital Narratives (IDN) by means of colours. The article uses goal models for expressing protostories. The core of the article consists of the case study where two colour synes-thetes were asked to choose colours for eight emotions. Thereafter the same synesthetes were asked to choose colours for the emotions that support the attainment of goals in the Cinderella narrative, which serves as an example protostory. The article also discusses the perspectives for applying the method proposed by us to using colours in constructing emotions by IDNs.

Smartphones are now ubiquitous, yet our understanding of user interface factors that maximize mobile user experience (UX), is still limited. This work presents a controlled experiment, which investigated factors that affect the usability and UX of a mobile incident reporting app. The results indicate that sequence of user interface elements matters while striving to increase UX, and that there is no difference between tab and scrolling as navigation modalities in short forms. These findings can serve as building blocks for empirically-derived guidelines for mobile incident reporting., Comment: Paper accepted for presentation at the 5th International Conference on Human Computer Interaction Theory and Applications (HUCAPP@VISIGRAPP 2021)