Search

Genomic research emerges from collaborative work within and across different scientific disciplines. A diverse range of visualisation techniques has been employed to aid this research, yet relatively little is known as to how these techniques facilitate collaboration. We conducted a case study of collaborative research within a biomedical institute to learn more about the role visualisation plays in genomic mapping. Interviews were conducted with molecular biologists (N = 5) and bioinformaticians (N = 6). We found that genomic research comprises a variety of distinct disciplines engaged in complex analytic tasks that each resist simplification, and their complexity influences how visualisations were used. Visualisation use was impacted by group-specific interactions and temporal work patterns. Visualisations were also crucial to the scientific workflow, used for both question formation and confirmation of hypotheses, and acted as an anchor for the communication of ideas and discussion. In the latter case, two approaches were taken: providing collaborators with either interactive or static imagery representing a viewpoint. The use of generic software for simplified visualisations, and quick production and curation was also noted. We discuss these findings with reference to group-specific interactions and present recommendations for improving collaborative practices through visual analytics.

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.

Researchers and software users benefit from the rapid growth of artificial intelligence (AI) to an unprecedented extent in various domains where automated intelligent action is required. However, as they continue to engage with AI, they also begin to understand the limitations and risks associated with ceding control and decision-making to not always transparent artificial computer agents. Understanding of “what is happening in the black box” becomes feasible with explainable AI (XAI) methods designed to mitigate these risks and introduce trust into human-AI interactions. Our study reviews the essential capabilities, limitations, and desiderata of XAI tools developed over recent years and reviews the history of XAI and AI in education (AIED). We present different approaches to AI and XAI from the viewpoint of researchers focused on AIED in comparison with researchers focused on AI and machine learning (ML). We conclude that both groups of interest desire increased efforts to obtain improved XAI tools; however, these groups formulate different target user groups and expectations regarding XAI features and provide different examples of possible achievements. We summarize these viewpoints and provide guidelines for scientists looking to incorporate XAI into their own work.

There are a growing number of jobs related to web development, yet there is little formal literature about the accessibility of web development with a screen reader. This article describes research to explore (1) web development accessibility issues and their impact on blind learners and programmers; (2) tools and strategies used to address issues; and (3) opportunities for creating inclusive web development curriculum and supportive tools. We conducted a Comprehensive Literature Review (CLR) to formulate accessibility issue categories, then interviewed 12 blind programmers to validate and expand on both issues in education and practice. The CLR yielded five issue categories: (1) visual information without an accessible equivalent, (2) orienting, (3) navigating, (4) lack of support, and (5) knowledge and use of supportive technologies. Our interview findings validated the use of CLR-derived categories and revealed nuances specific to learning and practicing web development. Blind web developers grapple with the inaccessibility of demonstrations and explanations of web design concepts, wireframing software, independent verification of computed Cascading Style Sheets (CSS), and navigating browser-based developer tool interfaces. Tools and strategies include seeking out alternative education materials to learn independently, use of CSS frameworks, collaboration with sighted colleagues, and avoidance of design and front-end development. This work contributes to our understanding of accessibility issues specific to web development and the strategies that blind web developers employ in both educational and applied contexts. We identify areas in which greater awareness and application of accessibility best practices are required in Web education, a need to disseminate existing screen reader strategies and accessible tools, and to develop new tools that support Web design and validation of CSS. Finally, this research signals future directions for the development of accessible web curriculum and supportive tools, including solutions that leverage artificial intelligence, tactile graphics, and supportive-online communities of practice.

There are many multiple-stage racing competitions in various sports such as swimming, running, or cycling. The wide availability of affordable tracking devices facilitates monitoring the position along with the race of all participants, even for non-professional contests. Getting real-time information of contenders is useful but also unleashes the possibility of creating more complex visualization systems that ease the understanding of the behavior of all participants during a simple stage or throughout the whole competition. In this paper we focus on bicycle races, which are highly popular, especially in Europe, being the Tour de France its greatest exponent. Current visualizations from TV broadcasting or real-time tracking websites are useful to understand the current stage status, up to a certain extent. Unfortunately, still no current system exists that visualizes a whole multi-stage contest in such a way that users can interactively explore the relevant events of a single stage (e.g. breakaways, groups, virtual leadership...), as well as the full competition. In this paper, we present an interactive system that is useful both for aficionados and professionals to visually analyze the development of multi-stage cycling competitions. This work has been supported in part by project TIN2017-88515-C2-1-R(GEN3DLIVE), from the Spanish Ministerio de Economía y Competitividad, by 839 FEDER (EU) funds and by the Grant No. PID2019-106426RB-C31 / AEI / 10.13039/501100011033 from the Spanish Government.

Air pollution is a global challenge for cities across the globe. Understanding the public perception of air pollution can help policymakers engage better with the public and appropriately introduce policies. Accurate public perception can also help people to identify the health risks of air pollution and act accordingly. Unfortunately, current techniques for determining perception are not scalable: it involves surveying few hundred people with questionnaire-based surveys. Using the advances in natural language processing (NLP), we propose a more scalable solution called Vartalaap to gauge public perception of air pollution via the microblogging social network Twitter. We curated a dataset of more than 1.2M tweets discussing Delhi-specific air pollution. We find that (unfortunately) the public is supportive of unproven mitigation strategies to reduce pollution, thus risking their health due to a false sense of security. We also find that air quality is a year-long problem, but the discussions are not proportional to the level of pollution and spike up when pollution is more visible. The information required by Vartalaap is publicly available and, as such, it can be immediately applied to study different societal issues across the world.

Usage of Twitter by politicians has become more prevalent in recent years, with a goal of influencing the electorate and public perception. We collect, explore, and analyze over 12 years of public Twitter interactions of U.S. senators and representatives. Using community detection algorithms on these interaction networks, and without considering the content of the tweets, we are able to infer the political affiliation of each member of Congress with up to 98.8% accuracy in the House and 94.1% accuracy in the Senate. In addition, we define two metrics that can determine the political ideology of members of Congress achieving a very high Spearman’s rank correlation of 0.86 with the existing DW-NOMINATE score from the field of political science. Finally, we expand our structural analysis to intra-party factions and found evidence that some factions act on Twitter more cohesively than others, suggesting an increasing risk of an echo chamber effect when promoting their political agenda.

Smart Home Personal Assistants (SPA) are an emerging innovation that is changing the way in which home users interact with the technology. However, there are a number of elements that expose these systems to various risks: i) the open nature of the voice channel they use, ii) the complexity of their architecture, iii) the AI features they rely on, and iv) their use of a wide-range of underlying technologies. This paper presents an in-depth review of the security and privacy issues in SPA, categorizing the most important attack vectors and their countermeasures. Based on this, we discuss open research challenges that can help steer the community to tackle and address current security and privacy issues in SPA. One of our key findings is that even though the attack surface of SPA is conspicuously broad and there has been a significant amount of recent research efforts in this area, research has so far focused on a small part of the attack surface, particularly on issues related to the interaction between the user and the SPA devices. We also point out that further research is needed to tackle issues related to authorization, speech recognition or profiling, to name a few. To the best of our knowledge, this is the first article to conduct such a comprehensive review and characterization of the security and privacy issues and countermeasures of SPA., Comment: Accepted for publication in ACM Computing Surveys

When discussing interpretable machine learning results, researchers need to compare them and check for reliability, especially for health-related data. The reason is the negative impact of wrong results on a person, such as in wrong prediction of cancer, incorrect assessment of the COVID-19 pandemic situation, or missing early screening of dyslexia. Often only small data exists for these complex interdisciplinary research projects. Hence, it is essential that this type of research understands different methodologies and mindsets such as the Design Science Methodology, Human-Centered Design or Data Science approaches to ensure interpretable and reliable results. Therefore, we present various recommendations and design considerations for experiments that help to avoid over-fitting and biased interpretation of results when having small imbalanced data related to health. We also present two very different use cases: early screening of dyslexia and event prediction in multiple sclerosis.

Social computing is ubiquitous and intensifying in the 21st Century. Originally used to reference computational augmentation of social interaction through collaborative filtering, social media, wikis, and crowdsourcing, here I propose to expand the concept to cover the complete dynamic interface between social interaction and computation, including computationally enhanced sociality and social science, socially enhanced computing and computer science, and their increasingly complex combination for mutual enhancement. This recommends that we reimagine Computational Social Science as Social Computing, not merely using computational tools to make sense of the contemporary explosion of social data, but also recognizing societies as emergent computers of more or less collective intelligence, innovation and flourishing. It further proposes we imagine a socially inspired computer science that takes these insights into account as we build machines not merely to substitute for human cognition, but radically complement it. This leads to a vision of social computing as an extreme form of human computer interaction, whereby machines and persons recursively combine to augment one another in generating collective intelligence, enhanced knowledge, and other social goods unattainable without each other. Using the example of science and technology, I illustrate how progress in each of these areas unleash advances in the others and the beneficial relationship between the technology and science of social computing, which reveals limits of sociality and computation, and stimulates our imagination about how they can reach past those limits together.

Touch-surface multi-stroke gestures, as well as freehand drawings, are typically acquired by devices and sensors as a suite of timestamped points on a plane. This Cartesian coordinate system, although useful for computation like complexity analysis, gesture classification and recognition, becomes complex and inefficient when gestures need to be visualized and directly manipulated for editing. To address these challenges, a new mathematical representation of these gestures via a Bézier curve is defined to initiate a model-based approach for gesture direct manipulation (e.g., cut, copy, paste, translate, scale, rotate, deform, crop, compose, decompose). SketchADoodle, an Android-based mobile application for drawing, gesturing, demonstrates how the pseudo-code of the Bézier-based operations are engineered for real-time direct manipulation. We release the programming code for further development of gesture-based user interfaces based on Bézier curves.

We investigate professional greenhouse growers’ user experience (UX) when using climate-management systems in their daily work. We build on the literature on UX, in particular UX at work, and extend it to ordinary UX at work . In a 10-day diary study, we collected data with a general UX instrument (AttrakDiff), a domain-specific instrument, and interviews. We find that AttrakDiff is valid at work; its three-factor structure of pragmatic quality (PQ), hedonic identification quality, and hedonic stimulation quality is recognizable in the growers’ responses. In this article, UX at work is understood as interactions among technology, tasks, structure, and actors. Our data support the recent proposal for the ordinariness of UX at work. We find that during continued use, UX at work is middle-of-the-scale, remains largely constant over time, and varies little across use situations. For example, the largest slope of the four AttrakDiff constructs when regressed over the 10 days was as small as 0.04. The findings contrast existing assumptions and findings in UX research, which is mainly about extraordinary and positive experiences. In this way, the present study contributes to UX research by calling attention to the mundane, unremarkable, and ordinary UXs at work.

People are becoming increasingly sophisticated in their ability to navigate information spaces using search, hyperlinks, and visualization. But, mobile phones preclude the use of multiple coordinated views that have proven effective in the desktop environment (e.g., for business intelligence or visual analytics). In this work, we propose to model information as multivariate heterogeneous networks to enable greater analytic expression for a range of sensemaking tasks while suggesting a new, list-based paradigm with gestural navigation of structured information spaces on mobile phones. We also present a mobile application, called Orchard, which combines ideas from both faceted search and interactive network exploration in a visual query language to allow users to collect facets of interest during exploratory navigation. Our study showed that users could collect and combine these facets with Orchard, specifying network queries and projections that would only have been possible previously using complex data tools or custom data science.

With the advent of pervasive usage of online platforms, online abusive behavior has become an indispensable part of our life demanding great attention from the research community. Accordingly, the research community is spending its effort on the demanding task, however, perhaps having much less effort on emails, even though emails are identified as a prominent source of exchanging online abusive behaviors. To fill in this gap in the literature, we conduct an in-depth study to investigate online abusive behavior having a special focus on emails. To do so, we perform a mixed-method user study consisting of formative interviews (n=15) and a survey (n=65) over user’s experience, coping strategies, etc., pertinent to online abuse in Bangladesh, especially focusing on abuse over emails. We also dig into users’ perspectives to analyze strengths and challenges associated with different types of abuse detection systems for online platforms, especially for emails. One of the noteworthy findings of our study is that there exists a significant demand for abuse detection systems over emailing platforms even after having a lesser frequency of abuse occurring over emails. Our findings also highlight a certain level of user preference for an automated abuse detection system potentially considering its more control and fewer privacy concerns to users, however, being challenged due to having the limitation of lesser ability to detect implicit abuse. We also identify several limiting factors associated with a human-moderator-based abuse detection system, including less comfort, less trust in different types of moderators, inhumane demands to the moderators, and time delay in detecting abuses. These findings point to opportunities for design interventions for hybrid abuse detection systems, which is the most preferred system to the users, to overcome all the limitations of automated and human-moderator-based systems.

We describe a mobile app for helping people manage their mental health, keeping in mind the unique challenges offered by the Indian context. One of the biggest challenges in India is the stigma around mental health and consequently the concerns and misconceptions about reaching out for professional help. Therefore, the proposed app focuses on tackling the barriers that people experience when reaching out for help and provides a free-of-cost platform for interactions with mental health professionals to alleviate any concerns and obtain an informed view. All content is developed by clinical psychologists and rather than replacing professional help, the app provides an example of a technology intervention for mental health that supplements existing infrastructures of mental healthcare.

Augmented Reality (AR) has been shown to enhance the data visualization and analysis process by supporting users in their immersive exploration of data in a real-world context. However, authoring such visualizations still heavily relies on traditional, stationary desktop setups, which inevitably separates users from the actual working space. To better support the authoring process in immersive environments, we propose the integration of spatially-aware mobile devices. Such devices also enable precise touch interaction for data configuration while lowering the entry barriers of novel immersive technologies. We therefore contribute an initial set of concepts within a scenario for authoring AR visualizations. We implemented an early prototype for configuring visualizations in-situ on the mobile device without programming and report our first impressions.

We must consider alternative displays for supporting productivity work in the context of an increasingly work-from-home world. Augmented reality virtual monitors can fulfill these needs by equipping users with large screen real estate, while maintaining portability, cost-effectiveness, and not occupying physical space. However, there are open questions regarding how to design virtual monitors. In my dissertation, I plan to investigate the design of virtual monitors to enhance productivity everywhere. This work comprises a group of design and user study contributions. I conducted a user study to understand the feasibility of virtual monitors and their tradeoffs when compared against physical monitors. I further propose investigating the design of static properties and dynamic behaviors that cannot be achieved through physical monitors.

Background Digital health research repositories propose sharing longitudinal streams of health records and personal sensing data between multiple projects and researchers. Motivated by the prospect of personalizing patient care (precision medicine), these initiatives demand broad public acceptance and large numbers of data contributors, both of which are challenging. Objective This study investigates public attitudes toward possibly contributing to digital health research repositories to identify factors for their acceptance and to inform future developments. Methods A cross-sectional online survey was conducted from March 2020 to December 2020. Because of the funded project scope and a multicenter collaboration, study recruitment targeted young adults in Denmark and Brazil, allowing an analysis of the differences between 2 very contrasting national contexts. Through closed-ended questions, the survey examined participants’ willingness to share different data types, data access preferences, reasons for concern, and motivations to contribute. The survey also collected information about participants’ demographics, level of interest in health topics, previous participation in health research, awareness of examples of existing research data repositories, and current attitudes about digital health research repositories. Data analysis consisted of descriptive frequency measures and statistical inferences (bivariate associations and logistic regressions). Results The sample comprises 1017 respondents living in Brazil (1017/1600, 63.56%) and 583 in Denmark (583/1600, 36.44%). The demographics do not differ substantially between participants of these countries. The majority is aged between 18 and 27 years (933/1600, 58.31%), is highly educated (992/1600, 62.00%), uses smartphones (1562/1600, 97.63%), and is in good health (1407/1600, 87.94%). The analysis shows a vast majority were very motivated by helping future patients (1366/1600, 85.38%) and researchers (1253/1600, 78.31%), yet very concerned about unethical projects (1219/1600, 76.19%), profit making without consent (1096/1600, 68.50%), and cyberattacks (1055/1600, 65.94%). Participants’ willingness to share data is lower when sharing personal sensing data, such as the content of calls and texts (1206/1600, 75.38%), in contrast to more traditional health research information. Only 13.44% (215/1600) find it desirable to grant data access to private companies, and most would like to stay informed about which projects use their data (1334/1600, 83.38%) and control future data access (1181/1600, 73.81%). Findings indicate that favorable attitudes toward digital health research repositories are related to a personal interest in health topics (odds ratio [OR] 1.49, 95% CI 1.10-2.02; P=.01), previous participation in health research studies (OR 1.70, 95% CI 1.24-2.35; P=.001), and awareness of examples of research repositories (OR 2.78, 95% CI 1.83-4.38; P Conclusions This study reveals essential factors for acceptance and willingness to share personal data with digital health research repositories. Implications include the importance of being more transparent about the goals and beneficiaries of research projects using and re-using data from repositories, providing participants with greater autonomy for choosing who gets access to which parts of their data, and raising public awareness of the benefits of data sharing for research. In addition, future developments should engage with and reduce risks for those unwilling to participate.

With the increasing number of flights in the recent years, airlines and aircraft manufacturers are facing a daunting problem: shortage of pilots. One solution to this is to reduce the number of pilots in the aircraft and move towards single pilot operations (SPO). However, with this approach, the safety and quality of the flights must be guaranteed. Due to the complex nature of piloting task, a form of human-machine teaming is required to provide extra help and insight to the pilot. To this end, it is natural to look for proper artificial intelligence (AI) solutions as the field has evolved rapidly through the past decades with rise of machine learning and deep learning. The ideal AI for this task should aim to improve the human decision-making and focus on interaction with human rather than simply automating processes without human intervention. This particular field of AI is designed to communicate with the human and is known as cognitive computing (CC). To this end, several technologies can be employed to cover different aspects of interaction. One such technology is augmented reality (AR) which as of today, has matured enough to be used in commercial products. As such, an experiment was conducted to study the interaction between the pilot and CC teammate, and understand whether assistance is required to enable safe transition towards SPO.

Immersive Virtual Reality (VR) can create compelling context-specific emotional experiences, but very few studies explore the importance of emotion-relevant contextual cues in VR. In this thesis, we will investigate how to use combined contextual and physiological cues to improve emotion recognition in VR with a view of enhancing shared VR experiences. We will explore how relatively low-cost sensors can capture physiological information and how this can be combined with user and environmental context cues to more accurately measure emotion. The main novelty of the research is that it will demonstrate the first Personalized Real-time Emotion-Adaptive Context-Aware VR (PerAffectly VR) system and provide significant insight into how to create, measure, and share emotional VR experiences. The research will be helpful in multiple domains such as tourism, training, entertainment, and gaming. It will also enable the creation of VR interfaces that automatically adapt to the user’s emotional needs and provide a better user experience.

We present the first deployment of an open, scalable and distributed spatial computing platform. We show in action its extensible protocols for discovering spatial services and spatial content in a geographic area, representing poses of real and virtual cameras and objects, exchanging content records, and interfacing with spatial computing services like visual localization. We also demonstrate the usage of the platform through a WebXR reference client in the first city-wide prototype testbed in Bari, Italy.

Finding a specific object is one of the basic tasks in both physical and virtual environments. However, visually scanning large scenes can be inefficient and frustrating, especially when the target is outside of one’s field of view. As a result, there have been several studies for supporting target finding tasks in a three-dimensional environment with various visual cues. However, most studies focused on conveying either the relative position or the distance of the target to users. In this study, we propose PanoCue, which is a visual cue that overlays a panorama view of the surroundings at the center of the user’s field of view for conveying both the position and the distance of a target in a 3D space with respect to the user’s location and head orientation. For evaluation, we conducted a user study with 20 participants where they were asked to find a target under different visual cue conditions: PanoCue, Radar, Arrow, and a baseline without cues. As a result, we found that the presence of visual cues improves task performance and that PanoCue significantly reduces the travel distance. Findings also showed that our feedback design received positive ratings in terms of easiness, fatigue, and satisfaction.

Interaction and augmentation strongly depend on the context of use and the respective human aspects. In this paper, we present the use of Mixed Reality (MR) for the creation of experience prototypes and the evaluation of mobile applications in a (simulated) usage context. The novelty is the integration of real mobile devices into MR according to the augmented virtuality principle and the mirroring of the screen from the real device to the virtual replica. This preserves the user experience and the full range of functions when using the mobile device in MR allowing it to be operated as in reality. In the MR environment it is possible to simulate usage contexts and to investigate the user experience with mobile applications. First, we show the technical concept and then we demonstrate the approach with a prototype smartphone app for a mobility assistant in two scenarios.

This paper presents new flick input interfaces to improve the usability of Japanese character input in VR space. We designed three different interfaces called TouchFlick, EyeFlick, and RoundFlick, which make use of various controller interactions and eye gestures. To investigate the effectiveness of these methods, we compared them with a conventional VR QWERTY keyboard with ray-based selection. We found that TouchFlick was significantly faster and RoundFlick had a significantly lower error rate for experienced users. On the other hand, the input efficiency was the same as that of the conventional method for those who had no experience with flick input. Regarding subjective evaluation, there was no significant difference in usability and mental workload.

We present Light4AR, a light source estimation solution based on the detection of real cast shadows in an image captured by a mobile device. Given the camera pose of the image and a local 3D model of the scene, the approach consists in analyzing shadows cast by a reference object onto its supporting plane to determine the 3D position and the intensity of multiple light sources. By creating virtual point lights based on the resulting parameters and adding them to the AR scene, all the virtual objects can be illuminated and cast virtual shadows consistent with the real environment lighting, thereby enhancing user experience and object presence. In addition to offering the ability to share results over multiple users while preserving device resources, the server-based GPU implementation provides results in interactive time and makes photorealistic AR experiences accessible to most mobile devices, not limiting them to recent ones or specific OS. The proposed approach requires small manual input, limited to placing in the scene a reference object of known geometry, then selecting a region including the shadows cast by this object. We show the potential of this approach on several challenging scenes with various lighting configurations and background textures.

A two-study approach was used to explore the effects of tactile feedback patterns on virtual reality gaming experiences of participants, including presence, flow, usability, emotion, technology adoption, and tactile sensation. In our main study, participants were instructed to play a virtual reality fighting game. Five experimental conditions with different tactile feedback patterns (no-tactile, random, fixed, discrete, and realistic tactile feedback) were examined. The results of our study indicated that: (1) tactile feedback patterns have no significant effect on flow and emotion, and (2) the no-tactile feedback condition was rated significantly lower by the participants than other tactile feedback conditions in terms of presence, usability, technology adoption, and tactile sensation. A follow-up study was then conducted to understand the different effects of realistic feedback in active versus passive fighting games. The results of our follow-up study indicated that participants rated tactile sensation and usability higher when under the passive gaming condition. We discuss our findings along with the study limitations and future research directions.

We present a concept and early prototype for exploring volumetric medical data, e.g., from MRI or CT scans, in head-mounted Augmented Reality (AR) with a spatially tracked tablet. Our goal is to address the lack of immersion and intuitive input of conventional systems by providing spatial navigation to extract arbitrary slices from volumetric data directly in three-dimensional space. A 3D model of the medical data is displayed in the real environment, fixed to a particular location, using AR. The tablet is spatially moved through this virtual 3D model and shows the resulting slices as 2D images. We present several techniques that facilitate this overall concept, e.g., to place and explore the model, as well as to capture, annotate, and compare slices of the data. Furthermore, we implemented a proof-of-concept prototype that demonstrates the feasibility of our concepts. With our work we want to improve the current way of working with volumetric data slices in the medical domain and beyond.

The eventual success of many AR and VR intelligent interactive systems relies on the ability to collect user motion data at large scale. Realistic simulation of human motion trajectories is a potential solution to this problem. Simulated user motion data can facilitate prototyping and speed up the design process. There are also potential benefits in augmenting training data for deep learning-based AR/VR applications to improve performance. However, the generation of realistic motion data is nontrivial. In this paper, we examine the specific challenge of simulating index finger movement data to inform mid-air gesture keyboard design. The mid-air gesture keyboard is deployed on an optical see-through display that allows the user to enter text by articulating word gesture patterns with their physical index finger in the vicinity of a visualized keyboard layout. We propose and compare four different approaches to simulating this type of motion data, including a Jerk-Minimization model, a Recurrent Neural Network (RNN)-based generative model, and a Generative Adversarial Network (GAN)-based model with two modes: style transfer and data alteration. We also introduce a procedure for validating the quality of the generated trajectories in terms of realism and diversity. The GAN-based model shows significant potential for generating synthetic motion trajectories to facilitate design and deep learning for advanced gesture keyboards deployed in AR and VR.

Attributions are a common local explanation technique for deep learning models on single samples as they are easily extractable and demonstrate the relevance of input values. In many cases, heatmaps visualize such attributions for samples, for instance, on images. However, heatmaps are not always the ideal visualization to explain certain model decisions for other data types. In this review, we focus on attribution visualizations for time series. We collect attribution heatmap visualizations and some alternatives, discuss the advantages as well as disadvantages and give a short position towards future opportunities for attributions and explanations for time series., 4 pages, 1 page refrences, TREX 2021: Workshop on TRust and EXpertise in Visual Analytics

The multivariate Spatio-Temporal data challenges the analysts along the navigation and exploration of the information, using Data Analytics techniques. We present SPLORT, as a user-centered web-based solution for segmented Visual Analytics. The tool involves a “What-Where-When” pyramidal paradigm to drive Spatio-temporal analysis towards the findings of new insights. It features multi-layered coordinated views to help the analyst navigate and explore multivariate data simultaneously. We evaluated this web-based tool, and reported the findings and results, considering users’ performance and perceptions, basing our analysis on their Likert scale (0-5) responses, regarding their experience and the tool's usability, and the sentiment of their feedback, by using Natural Language Processing (NLP) techniques. Our findings from the pilot study held, suggest that SPLORT successfully allows the users to visually query, explore and analyze contextless multi-scale attributes within space and time, which quickly drives them to new insights and to support easier decision-making processes. Moreover, the tool's flexibility, due to its multi-layered GUI, supports better and more friendly visual analytics.

Consuming video content poses significant challenges for many screen magnifier users, which is the "go to" assistive technology for people with low vision. While screen magnifier software could be used to achieve a zoom factor that would make the content of the video visible to low-vision users, it is oftentimes a major challenge for these users to navigate through videos. Towards making videos more accessible for low-vision users, we have developed the SViM video magnifier system [6]. Specifically, SViM consists of three different magnifier interfaces with easy-to-use means of interactions. All three interfaces are driven by visual saliency as a guided signal, which provides a quantification of interestingness at the pixel-level. Saliency information, which is provided as a heatmap is then processed to obtain distinct regions of interest. These regions of interests are tracked over time and displayed using an easy-to-use interface. We present a description of our overall design and interfaces.

Based on an Artificial Neural Network model, Cell Assembly-based Task Analysis is a new method that outputs a task performance model composed of integrated mind-brain Cell Assemblies, which are currently believed to be the most plausible, general organisation of the brain and how it supports mental operations. A simplified model of Cell Assemblies and their cognitive architecture is described and then used in the method. A brief sub-task is analysed. The method’s utility to research in Artificial Intelligence, neuroscience and cognitive psychology is discussed and the possibility of a General Theory suggested.

In our daily life, it is inevitable to confront the condition which we feel confident or unconfident. Under these conditions, we might have different expressions and responses. Not to mention under the situation when a human communicates with a robot. It is necessary for robots to behave in various styles to show adaptive confidence degree, for example, in previous work, when the robot made mistakes during the interaction, different certainty expression styles have shown influence on humans’ truthfulness and acceptance. On the other hand, when human feel uncertain on the robot’s utterance, the approach of how the robot recognizes human’s uncertainty is crucial. However, relative researches are still scarce and ignore individual characteristics. In current study, we designed an experiment to obtain human verbal and non-verbal features under certain and uncertain condition. From the certain/uncertain answer experiment, we extracted the head movement and voice factors as features to investigate if we can classify these features correctly. From the result, we have found that different people had distinct features to show different certainty degree but some participants might have a similar pattern considering their relatively close psychological feature value. We aim to explore different individuals’ certainty expression patterns because it can not only facilitate humans’ confidence status detection but also is expected to be utilized on robot side to give the proper response adaptively and thus spice up the Human-Robot Interaction.

Relatively little work has empirically examined use and non-use of social technologies as more than a dichotomous binary, despite increasing calls to do so. This paper compares three different forms of non/use that might otherwise fall under the single umbrella of Facebook "user": (1) those who have a current active account; (2) those who have deactivated their account; and (3) those who have considered deactivating but not actually done so. A subset of respondents (N=256) from a larger, demographically representative sample of internet users completed measures for usage and perceptions of Facebook, Facebook addiction, privacy experiences and behaviors, and demographics. Multinomial logistic regression modeling shows four specific variables as most predictive of a respondent's type: negative effects from "addictive" use, subjective intensity of Facebook usage, number of Facebook friends, and familiarity with or use of Facebook's privacy settings. These findings both fill gaps left by, and help resolve conflicting expectations from, prior work. Furthermore, they demonstrate how valuable insights can be gained by disaggregating "users" based on different forms of engagement with a given technology.

With the development of wireless communications and networks, HCC (human-centred computing) has attracted considerable attention in recent years throughout the medical field. HCC can provide an effective integration of various medical auxiliary diagnosis models using machine learning algorithms. In medical HCC, deep learning has demonstrated its powerful ability in the field of computer vision. However, image processing based on deep learning usually requires a large amount of labeled data, which requires significant resources since it needs to be completed by doctors, and it is difficult to collect a large amount of data for some rare diseases. Therefore, how to use the deep learning method to obtain an effective auxiliary diagnosis model based on a small sample or zero sample data set has become an important issue in the study of medical auxiliary diagnosis. We proposes an auxiliary diagnosis model acquisition method based on a variational auto-encoder and zero sample augmentation technology, and the incremental update training program based on wireless communications and networks is designed to obtain the auxiliary diagnosis model to solve the difficulty of collecting a large amount of valid data. The experimental results show that the model obtained by the above method based on a small sample or zero sample data set can effectively diagnose the types of skin diseases, which helps doctors make better judgments.

Graphical Adaptive Menus are Graphical User Interface menus whose predicted items of immediate use can be automatically rendered in a prediction window. Rendering this prediction window is a key question for adaptivity to enable the end-user to efficiently differentiate predicted items from normal ones and to consequently select appropriate items. Adaptivity for graphical menus has been investigated more for normal screens, such as desktops, than for small screens, such as smartphones, where real estate imposes severe rendering constraints. To address this question, this article defines and explores a design space where graphical adaptive menus are structured based on Bertin’s eight visual variables (i.e., position, size, shape, value, color, orientation, texture, and motion) and their combination by comparing their rendering for small screens with respect to normal screens. Based on this design space, previously introduced graphical adaptive menus are revisited in terms of four stability properties (i.e., spatial, physical, format, and temporal), and new menu designs are introduced and discussed for both normal and small screens. The resulting set of graphical adaptive menu has been subject to a preference analysis from which a particular design emerged: the cloud menu, where predicted items are arranged in an adaptive tag cloud. We investigate empirically the effect of the cloud menu on the item selection time and the error rate with respect to a static menu and an adaptive linear menu. This article then suggests a set of usability guidelines for designers and practitioners to design graphical adaptive menus in general and cloud menus in particular.

Visualizing contextual information to a map often comes at the expense of overplotting issues. Especially for use cases with relevant map features in the immediate vicinity of an information to add, occlusion of the relevant map context should be avoided. We present SurgeryCuts, a map manipulation technique for the creation of additional canvas area for contextual visualizations on maps. SurgeryCuts is occlusion-free and does not shift, zoom or alter the map viewport. Instead, relevant parts of the map can be cut apart. The affected area is controlledly distorted using a parameterizable warping function fading out the map distortion depending on the distance to the cut. We define extended metrics for our approach and compare to related approaches. As well, we demonstrate the applicability of our approach at the example of tangible use cases and a comparative user study. published

The role of emerging technologies in the rehabilitative care of patients continues to grow. In order to effectively provide care in a complex medical system, rehabilitative specialists need to be fluent in how technology can enhance the delivery of patient care. In concert, technology experts need to understand how patients are affected by their technological designs. Currently, there is an absence of data to evaluate the impact of combining these two professional groups (physical therapy and human centered computing) to serve the needs of rehabilitative patients. The purpose of this project was to enhance the ability for both rehabilitative professionals and human centered computing specialists to address the healthcare needs of individuals requiring adaptive equipment. An interprofessional educational series was developed to integrate the knowledge and skills of Doctor of Physical Therapy students with students from a human centered computing program to develop adaptive equipment personalized to individual patients using 3D printing. The pilot program was evaluated using survey data and interviews collected from both student and patient perspectives. This paper contributes to the understanding of challenges to unique interprofessional collaborations and provides suggestions to foster more effective educational training for both disciplines. Strategies for improving the development of patient-centered 3D printed adaptive equipment as well as the practical application of using 3D printed assistive technology (AT) in the delivery of client care are discussed.

This study was supported by the Office of Sponsored Research (OSR) at King Abdullah University of Science and Technology (KAUST) under the “Virtual Red Sea Initiative” (award #REP/1/3268-01-01). We want to acknowledge Dr. Hari Prasad Dasari for his help and contribution in organizing the interviews with domain experts. We also like to thank KAUST Visualization Core Lab for their help and support.

Many public buildings are entered through reception areas, intended for visitors to sit and wait in to be met. A concern is how to make visitors feel welcome while in transit. Hospitals, medical centres and other healthcare organisations are a special case where the challenge is to enable patients and families feel less anxious when waiting. One approach has been to design for distraction – where displays, surfaces, and interactive installations are created to draw visitor's attention away from their immediate thoughts. However, little is known as to how people respond to such interventions. We present the findings of an ethnographic study that examined the social and psychological effects of using distraction as a design principle in a children's hospital reception area. We discuss the challenges of designing to distract, in relation to how it can be combined with other architectural and HCI ones, when developing new human–building interfaces.

Conventional online learning typically allows an instructor to deliver instruction to students via a predefined curriculum and within a fixed knowledge structure (i.e., explaining the instructional subject). With the dramatic growth of social media technology and correlated data aggregation, some sort of instant knowledge is obtained by daily users. An emerging type of knowledge (i.e., social knowledge) has been identified and may lead to self-paced learning from social networks, which is simply defined as social learning. This article points out three important issues for social learning, namely, knowledge retrieval via temporal social factors, and the connection between social network and the knowledge domain. Two significant automation mechanisms, lecture generation for self-regulated learning and influencing domain computation for opportunity finding, are suggested to facilitate the process of social learning. A prototype system based on Elgg was implemented, sourced by a federated repository that has stored and shared more than 1.5 millions transactions (e.g., content, interactions, etc.). We conclude that timely social knowledge (or crowdsourcing results) can be widely applied in the next era of online learning environment. Findings through the statistical analysis are prospective to support understanding of phenomenon of social learning and design of future learning platform for followup researchers.

In this paper, we present a behavioral biometric-based smartphone user authentication mechanism, namely, AnswerAuth , which relies on the very common users’ behavior. Behavior, here, refers to the way a user slides the lock button on the screen, to unlock the phone, and brings the phone towards her ear. The authentication mechanism works with the biometric behavior based on the extracted features from the data recorded using the built-in smartphone sensors, i.e., accelerometer, gyroscope, gravity, magnetometer and touchscreen, while the user performed sliding and phone-lifting actions. We tested AnswerAuth on a dataset of 10,200 behavioral patterns collected from 85 users while they performed the unlocking actions, in sitting, standing, and walking postures, using six state-of-the-art conceptually different machine learning classifiers in two settings, i.e., with and without simultaneous feature selection and classification. Among all the chosen classifiers, Random Forest (RF) classifier proved to be the most consistent and accurate classifier on both full and reduced features and provided a True Acceptance Rate (TAR) as high as 99.35%. We prototype proof-of-the-concept Android app, based on our findings, and evaluate it in terms of security and usability. Security analysis of AnswerAuth confirms its robustness against the possible mimicry attacks. Similarly, the usability study based on Software Usability Scale (SUS) 1 questionnaire verifies the user-friendliness of the proposed scheme (SUS Score of 75.11). Experimental results prove AnswerAuth as a secure and usable authentication mechanism.

People with visual impairments may find it hard to navigate in unfamiliar spaces. Even though several systems have been proposed to make navigation easier, such systems often ignore two essential factors for anyone using a support system: comfort and portability. This work presents a portable navigation support system for people with visual impairments. The system utilizes Smartphones as a portable platform and leverages the application of a deep learning object detection system for guiding the user during navigation. Audio output is used to communicate information about obstacles to the user. The system offers promising accuracy in object detection and distance estimation.