Search

Virtual reality as a platform provides untapped potential for the popular music video through its affordance of multisensorial convergence, which aligns with and greatly expands upon the evolution of music as an audiovisual medium. Current popular models for music video representation in virtual spaces utilize many of the unique features afforded by virtual reality including 360 video, audiovisual reactivity, spatialization and interactivity, however, most still act as repeatable experiences tethered to design conventions for two-dimensional digital spaces and ideals related to the archetypal performer-audience dynamic.This paper explores the potential for music artists to create toward virtual reality by examining the intersection between narrative-based music production and existing music-centered virtual reality experiences, proposing a framework in which emotion, narrative and both mediums iteratively inform each other to create a set of altered design principles for meaningful creation toward this intersectional space.A series of music-centered virtual reality case studies will be presented to support this argument. Exemplified will be the interactive virtual reality composition “Pigments of Imagination”, a musical piece created by the author with virtual reality as a destination, and thus presented as a narrative journey weaved by and dependent on the technical relationship between compositional, visual, and interactive elements in a virtual space.

The growing role of technology in society has led to calls for more experiential, community-engaged learning to equip students with the skills and qualities needed for developing technology for the public interest. Yet, despite the growing number of initiatives that seek to provide such experiences, there is still limited research on the barriers that make it challenging to do so, particularly from the perspective of community members not currently engaged in campus partnerships, but interested in doing so. To better understand these challenges, we ran a matchmaking program to help community members connect with potential campus partners and conducted a series of interviews with 8 community stakeholders (nonprofits, government programs, and small tech consultancies) to understand potential project needs, expectations for collaboration, and views on connecting community needs to student learning. We analyzed these interviews to understand barriers and identified three themes around the central concept of misalignments: misalignments between the professional and non-profit world versus the educational system, misalignments between the goal of delivering project impact versus the goal of completing a course or developing a career, and misalignments between expectations of professionalism versus the reality of volunteers and learners. We conclude by reflecting on approaches to overcoming these misalignments and implications for designing computational ecosystems that support educational experiences more integrated with the community

With the recent development of the Virtual Reality (VR) industry, the increasing number of VR users pushes the demand for the massive production of immersive and expressive VR scenes in related industries. However, creating expressive VR scenes involves the reasonable organization of various digital content to express a coherent and logical theme, which is time-consuming and labor-intensive. In recent years, Large Language Models (LLMs) such as ChatGPT 3.5 and generative models such as stable diffusion have emerged as powerful tools for comprehending natural language and generating digital contents such as text, code, images, and 3D objects. In this paper, we have explored how we can generate VR scenes from text by incorporating LLMs and various generative models into an automated system. To achieve this, we first identify the possible limitations of LLMs for an automated system and propose a systematic framework to mitigate them. Subsequently, we developed Text2VRScene, a VR scene generation system, based on our proposed framework with well-designed prompts. To validate the effectiveness of our proposed framework and the designed prompts, we carry out a series of test cases. The results show that the proposed framework contributes to improving the reliability of the system and the quality of the generated VR scenes. The results also illustrate the promising performance of the Text2VRScene in generating satisfying VR scenes with a clear theme regularized by our well-designed prompts. This paper ends with a discussion about the limitations of the current system and the potential of developing similar generation systems based on our framework. © 2024 IEEE.

Augmented Reality (AR) teleconferencing aims to enable remotely separated users to meet with each other in their own physical spaces as if they are face-to-face. Among all solutions, the video-avatar-based approach has the advantage of balancing fidelity and the sense of co-presence using easy-to-setup devices, including only a camera and an AR Head-Mounted Display (HMD). However, non-verbal cues indicating 'who is looking at whom' are always lost or misdelivered in multiparty teleconferencing experiences. To make users aware of such non-verbal cues, existing solutions explore screen-based visualizations, incorporate additional hardware, or alter to use a virtual avatar representation. However, they lack immersion, are less feasible for everyday usage due to complex installations, or lose the fidelity of remote users' authentic appearances. In this paper, we decompose such attention awareness into the awareness of being looked at and the awareness of attention between other users and address them in a decoupled process. Specifically, through a user study, we first find an unobtrusive and reasonable layout 'Attention Circle' to retarget a looker's head gaze to the one being looked at. We then conduct the second user study to find an effective and intuitive 'rotatable 2.5D video avatar with attention thumbnail' visualization to aid users in being aware of other users' attention. With the design choice distilled from the studies, we implement A3RT, a proof-of-concept prototype system that empowers attention-aware 2.5D-video-avatar-based multiparty AR teleconferencing in an easy, everyday setup. Ablation and usability studies on the prototype verify the effectiveness of our proposed components and the full system.

This article introduces an innovative Mixed Reality (MR) system designed explicitly for calligraphy learning and practice. Learners must prepare numerous copies of calligraphy works and character templates in traditional calligraphy study and practice. They often rely on oral guidance from teachers in person for their calligraphy practice. However, with the progress of digital technology, we envision leveraging MR wearable glasses combined with image capture and analysis techniques to assist calligraphy learners in improving their practice in a more flexible time. © 2024 IEEE.

Virtual reality (VR) is an immersive medium that offers users a unique opportunity to experience a digital environment realistically. As the demand for VR content continues to grow, the importance of effective VR editing techniques becomes increasingly apparent. This paper is a pioneering work investigating the effects of jump cuts on the viewer's sense of presence, viewing experience, and edit quality in cinematic VR. Specifically, this work focuses on using the 30-degree and 180-degree rules in VR editing to minimize the adverse effects of jump cuts. We conducted a user study with thirteen participants, who watched nine different VR edits and completed a survey for each edited video. Our results indicate that employing the 30-degree and 180-degree rules in VR editing can significantly improve the sense of presence, viewing experience, and edit quality while mitigating the negative effects of jump cuts. We provide valuable insights for VR content creators and editors to achieve more effective and immersive VR experiences. © 2024 IEEE.

Industry 5.0 marks the next stage of industrial revolutions, seamlessly integrating human capabilities with advanced technologies such as artificial intelligence, robotics, and the Internet of Things. This transformative approach emphasizes human–machine collaboration to enhance work quality and improve customer experiences. The pivotal technologies driving Industry 5.0, including smart manufacturing, advanced automation, and data analytics, are thoroughly explored. The chapter also investigates key enabling technologies for Industry 5.0, along with the concept of the Metaverse. Furthermore, it presents and discusses technical frameworks that offer implementation perspectives on Society 5.0 and the digital technologies of Industry 4.0. The comprehensive analysis sheds light on the potential benefits, challenges, and future trends of Industry 5.0, highlighting its profound impact on industries, the workforce, and society as a whole., QC 20240926

Understanding and recognizing emotions are important and challenging issues in the metaverse era. Understanding, identifying, and predicting fear, which is one of the fundamental human emotions, in virtual reality (VR) environments plays an essential role in immersive game development, scene development, and next-generation virtual human-computer interaction applications. In this article, we used VR horror games as a medium to analyze fear emotions by collecting multi-modal data (posture, audio, and physiological signals) from 23 players. We used an LSTM-based model to predict fear with accuracies of 65.31% and 90.47% under 6-level classification (no fear and five different levels of fear) and 2-level classification (no fear and fear), respectively. We constructed a multi-modal natural behavior dataset of immersive human fear responses (VRMN-bD) and compared it with existing relevant advanced datasets. The results show that our dataset has fewer limitations in terms of collection method, data scale and audience scope. We are unique and advanced in targeting multi-modal datasets of fear and behavior in VR stand-up interactive environments. Moreover, we discussed the implications of this work for communities and applications. The dataset and pre-trained model are available at https://github.com/KindOPSTAR/VRMN-bD., Part of ISBN 9798350374025QC 20240514

Feminist self-defense is a form of victim prevention training with a plethora of positive physical, mental, and social outcomes. In Human-Computer Interaction (HCI) self-defense is still a relatively under-researched topic. As such, inspired by HCI’s recent interest in feminist causes and corporeal practices, the goal of the study is to explore how embodied interaction design can empower beginner self-defense practitioners. To do so, the study was conducted via two methods: semi-structured interviews with students and teachers, and a participatory speculative design workshop with novice practitioners. This resulted in several speculative design concepts based on the use of felt experiences as a design resource. The concepts demonstrate how design can be used as a vehicle for imagining feminist technology that challenges gender norms and plays the role of scaffolding for cultivating embodied empowerment.

We present DatAR, an Augmented Reality prototype designed to support neuroscientists in finding fruitful directions to explore in their own research. DatAR provides an immersive analytics environment for exploring relations between topics published in the neuroscience literature. Neuroscientists need to analyse large numbers of publications in order to understand whether a potential experiment is likely to yield a valuable contribution. Using a user-centred design approach, we have identified useful tasks in collaboration with neuroscientists and implemented corresponding functionalities in DatAR. This facilitates querying and visualising relations between topics. Participating neuroscientists have stated that the DatAR prototype assists them in exploring and visualising seldom-mentioned direct relations and also indirect relations between brain regions and brain diseases. We present the latest incarnation of DatAR and illustrate the use of the prototype to carry out two realistic tasks to identify fruitful experiments.

Traveling by public transport can be challenging for a visually impaired traveler. However, visual information can be supported by sonification, the use of non-speech sound to convey information about data. This research project aims to explore how sonification can be used to provide information to a traveler at a bus stop. Three situations are described together with different sonification design approaches that will later be further developed and evaluated.

This poster presents initial design steps exploring how sonification can be used to support visualization for comprehension of space and time in astronomical data. Radio signals travel at the speed of light. With a visualization of the universe, it is possible to travel faster than light and pass the radio waves leaving earth. We can then travel back in time. We propose to use sonification consisting of songs representing each year as a musical journey through space and time to create an engaging experience.

Feminist self-defense is a form of victim prevention training with a plethora of positive physical, mental, and social outcomes. In Human-Computer Interaction (HCI) self-defense is still a relatively under-researched topic. As such, inspired by HCI’s recent interest in feminist causes and corporeal practices, the goal of the study is to explore how embodied interaction design can empower beginner self-defense practitioners. To do so, the study was conducted via two methods: semi-structured interviews with students and teachers, and a participatory speculative design workshop with novice practitioners. This resulted in several speculative design concepts based on the use of felt experiences as a design resource. The concepts demonstrate how design can be used as a vehicle for imagining feminist technology that challenges gender norms and plays the role of scaffolding for cultivating embodied empowerment.

Virtual Reality (VR) broadcasting has seen widespread adoption in major sports events, attributed to its ability to generate a sense of presence, curiosity, and excitement among viewers. However, we have noticed that still shots reveal a limitation in the movement of VR cameras and hinder the VR viewing experience in current VR sports broadcasts. This paper aims to bridge this gap by engaging in a quantitative user analysis to explore the design and impact of dynamic VR shots on viewing experiences. We conducted two user studies in a digital hockey game twin environment and asked participants to evaluate their viewing experience through two questionnaires. Our findings suggested that the viewing experiences demonstrated no notable disparity between still and moving shots for single clips. However, when considering entire events, moving shots improved the viewer's immersive experience, with no notable increase in sickness compared to still shots. We further discuss the benefits of integrating moving shots into VR sports broadcasts and present a set of design considerations and potential improvements for future VR sports broadcasting., Part of proceedings ISBN 9798350328387QC 20240108

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training., The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/No changes have been made to the publication.

The majority of music consumption nowadays takes place on music streaming platforms. Whichever artists, albums, or songs are exposed to consumers on these platforms therefore greatly influences what music is ultimately consumed. As a result, the impact of these platforms on artists—their main item providers—is considerable. The recommender systems at the core of streaming platforms, though, have traditionally been developed focusing on end consumer objectives. Only recently, researchers have started to include item provider objectives, though rarely through reaching out to item providers directly. By omitting this important stakeholder’s point of view, we risk not understanding what artists value most, and might miss first-hand ideas on how to improve streaming platforms and recommender systems. Therefore, we conducted semi-structured interviews to capture the artists’ view. Specifically, we explore artists’ considerations regarding fairness, transparency, and diversity in music recommender systems, and the role artists envision for streaming platforms regarding those topics. We identify some topics with a clear consensus among artists, such as desiring more control over which music is recommended to whom, and expecting streaming platforms to actively increase music diversity in recommendations. In contrast, artists’ opinions differ on whether platforms should actively intervene in recommender systems to, e.g., increase localization or gender balance. Further, we observe that artists often take user preferences into account and even suggest new platform functionality to benefit both users and item providers. We encourage utilizing these insights when designing and evaluating music streaming platforms and recommender systems.

Physical distancing is a key measure to slow the spread of many highly infectious diseases, e.g. COVID-19. Streetscape interventions such as pedestrian signage can contribute to ensuring distances are kept, but it is unclear to what extent people comply with these in practice. This paper tackles this question using an immersive video environment to realistically simulate real-life streetscapes in the lab. In a controlled user study, we augmented panoramic video footage with pedestrian one-way street signage and recorded route decisions to assess compliance with distance keeping measures. Our results indicate that such signage affects routing decisions and can thus help pedestrians to avoid crowded situations where distance keeping is difficult. We also identified further factors affecting decisions and a correlation between intention to comply and actual compliance. The experimental method we used enabled us to effectively and safely carry out a study of a phenomenon that in the real world depends on interaction with the physical environment. This method may have applications in other areas in which simulations of physical environments are important.

Pedestrians, especially those with disabilities, rely on mobile map applications to plan their daily trips and navigate unfamiliar spaces. Yet, many of these applications do not provide crucial real-time information for pedestrians, including foot traffic, semipermanent and permanent obstacles, sidewalk accidents, and other barriers common to pedestrian spaces. Over the past decade, researchers and engineers in academia and industry have explored accessible navigation in a variety of mobile applications. Despite significant effort, accessible navigation features can only provide limited real-time information to select major metropolitan areas. One substantial obstacle preventing real-time accessible navigation from being more informative and deployed to more places is the expensive and manual process of regularly collecting and updating pedestrian walkway data. This thesis presents an initial feasibility study with eight student volunteers who commuted with scooters regularly at the University of California, Irvine. Data collected through custom GPS modules and the follow-up survey revealed insights about the plausibility of extracting real-time pedestrian walkway accessibility information from scooter riders’ travel patterns. My work calls for future researchers working on accessible maps to delve deeper into travel patterns of different human-controlled or autonomous wheeled devices, not just wheelchairs, on pedestrian walkways. Only when pedestrian walkway data collection becomes less manual and costly can updates happen often and more areas be covered.

User studies are a useful tool for researchers, allowing them to collect data on how users perceive, interact with and process different types of sensory information. If planned in advance, user experiments can be leveraged in every stage of a research project, from early design, prototyping and feature exploration to applied proofs of concept, passing through validation and data collection for model training. User studies can provide the researcher with different types of information depending on the chosen methodology: user performance metrics, surveys and interviews, field studies, physiological data, etc. Considering human perception and other cognitive processes is particularly important in computer graphics, where most research produces outputs whose ultimate purpose is to be seen or perceived by a human. Being able to measure in an objective and systematic way how the information we generate is integrated into the representational space humans create to situate themselves in the world means that researchers will have more information to implement optimal algorithms, tools and techniques. In this tutorial we will give an overview of good practices for user studies in computer graphics with a particular focus on virtual reality use cases. We will cover the basics on how to design, carry out and analyze good user studies, as well as different particularities to be taken into account in immersive environments., Peer Reviewed, Postprint (published version)

As robots have become more pervasive in our everyday life, social aspects of robots have attracted researchers' attention. Because emotions play a key role in social interactions, research has been conducted on conveying emotions via speech, whereas little research has focused on the effects of non-speech sounds on users' robot perception. We conducted a within-subjects exploratory study with 40 young adults to investigate the effects of non-speech sounds (regular voice, characterized voice, musical sound, and no sound) and basic emotions (anger, fear, happiness, sadness, and surprise) on user perception. While listening to the fairytale with the participant, a humanoid robot (Pepper) responded to the story with a recorded emotional sound with a gesture. Participants showed significantly higher emotion recognition accuracy from the regular voice than from other sounds. The confusion matrix showed that happiness and sadness had the highest emotion recognition accuracy, which aligns with the previous research. Regular voice also induced higher trust, naturalness, and preference compared to other sounds. Interestingly, musical sound mostly showed lower perceptions than no sound. A further exploratory study was conducted with an additional 49 young people to investigate the effect of regular non-verbal voices (female voices and male voices) and basic emotions (happiness, sadness, anger, and relief) on user perception. We also further explored the impact of participants' gender on emotion and social perception toward robot Pepper. While listening to a fairy tale with the participants, a humanoid robot (Pepper) responded to the story with gestures and emotional voices. Participants showed significantly higher emotion recognition accuracy and social perception from the voice + Gesture condition than Gesture only conditions. The confusion matrix showed that happiness and sadness had the highest emotion recognition accuracy, which aligns with the previous research. Interesti

Creating highly realistic Virtual Reality (VR) bicycle experiencescan be time-consuming and expensive. Moreover, it is unclear whathardware parts are necessary to design a bicycle simulator andwhether a bicycle is needed at all. In this paper, we investigatedcycling fdelity and control of VR bicycle simulators. For this, we developed and evaluated three cycling simulators: (1) cycling withouta bicycle (bikeless), (2) cycling on a fxed (stationary) and (3) movingbicycle (tandem) with four levels of control (no control, steering,pedaling, and steering + pedaling). To evaluate all combinations offdelity and control, we conducted a controlled experiment (N = 24)in indoor and outdoor settings. We found that the bikeless setupprovides the highest feeling of safety, while the tandem leads to thehighest realism without increasing motion sickness. Moreover, wediscovered that bicycles are not essential for cycling in VR., Part of proceedings ISBN 9781450394215QC 20231031

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training., The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/No changes have been made to the publication.

High-dimensional images (i.e., with many attributes per pixel) are commonly acquired in many domains, such as geosciences or systems biology. The spatial and attribute information of such data are typically explored separately, e.g., by using coordinated views of an image representation and a low-dimensional embedding of the high-dimensional attribute data. Facing ever growing image data sets, hierarchical dimensionality reduction techniques lend themselves to overcome scalability issues. However, current embedding methods do not provide suitable interactions to reflect image space exploration. Specifically, it is not possible to adjust the level of detail in the embedding hierarchy to reflect changing level of detail in image space stemming from navigation such as zooming and panning. In this paper, we propose such a mapping from image navigation interactions to embedding space adjustments. We show how our mapping applies the "overview first, details-on-demand" characteristic inherent to image exploration in the high-dimensional attribute space. We compare our strategy with regular hierarchical embedding technique interactions and demonstrate the advantages of linking image and embedding interactions through a representative use case., Computer Graphics and Visualisation, Pattern Recognition and Bioinformatics

As visualization makes the leap to mobile and situated settings, where data is increasingly integrated with the physical world using mixed reality, there is a corresponding need for effectively managing the immersed user's view of situated visualizations. In this paper we present an analysis of view management techniques for situated 3D visualizations in handheld augmented reality: a shadowbox, a world-in-miniature metaphor, and an interactive tour. We validate these view management solutions through a concrete implementation of all techniques within a situated visualization framework built using a web-based augmented reality visualization toolkit, and present results from a user study in augmented reality accessed using handheld mobile devices.

With the rapid advances in single-cell sequencing technologies, novel types of studies into the cell-type makeup of the brain have become possible. Biologists often analyze large and complex single-cell transcriptomic datasets to enhance knowledge of the intricate features of cellular and molecular tissue organization. A particular area of interest is the study of whether cell types and their gene regulation are conserved across species during evolution. However, in-depth comparisons across species of such high-dimensional, multi-modal single-cell data pose considerable visualization challenges. This paper introduces Cytosplore Simian Viewer, a visualization system that combines various views and linked interaction methods for comparative analysis of single-cell transcriptomic datasets across multiple species. Cytosplore Simian Viewer enables biologists to help gain insights into the cell type and gene expression differences and similarities among different species, particularly focusing on comparing human data to other species. The system validation in discovery research on real-world datasets demonstrates its utility in visualizing valuable results related to the evolutionary development of the middle temporal gyrus., Computer Graphics and Visualisation

In the HCI community, there is more openness and interest toward different forms of research outcomes beyond written academic publications. These include pictorial papers, video/audio documentaries, public exhibitions, posters and brochures, design fiction, comics, podcasts and many more. These alternative research outcomes play a critical role in explaining, disseminating, and translating valuable insights and knowledge from HCI research to people outside academic communities. We propose this workshop to initiate the conversation among researchers in the DIS community in generating alternative forms of research outcomes. What inspirations, motivations and critical factors influence the creation of alternative research outcomes? Who is the main audience, and what are the barriers and limitations of making them? The outcome of the workshop will be an enhanced understanding related to how HCI knowledge can be translated to or created for different audiences outside of academia, and a guide for HCI researchers towards creating alternate research outcomes., Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Human Information Communication Design

In recent years, there has been a notable proliferation and diversification of works in HCI, that integrate living microorganisms; an imperative lifeform dominating ecosystems of our planet. Yet despite the growing interest, there is a lack of structured lenses with which designers can strategize their processes of surfacing livingness; a material quality inherent in living artefacts with a potential to enrich user experiences and to initiate mutualistic care between humans and microorganisms. Through a systematic artefacts review and a case study on Flavobacteria, we have developed and instantiated a Taxonomy of Surfacing Livingness in Microbial Displays, consisting of six microbe-sensitive, tuneable mechanisms for human noticing of microorganisms: 1) Canvassing, 2) Marking, 3) Magnifying, 4) Translating, 5) Nudging, and 6) Molecular Programming. The taxonomy invites diverse and adaptable ways of generating and crafting microbial displays; towards overcoming microbe-specific surfacing constraints, integrating diverse stakeholders' values, and enabling nuanced address of microbial welfare., Materials and Manufacturing, Mechatronic Design, Emerging Materials

Advances in neurotechnology have enhanced and simplified our ability to research brain activity with low-cost and effective equipment. One such scalable and noninvasive technique is Electroencephalography (EEG), which detects and records electrical brain activity. Brain activity recognition is one of the emerging problems as EEG wearables become more readily available. Our research has modeled EEG signals to classify three states (i) music listening, (ii) movie watching, and (iii) meditating. The datasets incorporating the brain signals induced while performing these activities are NMED-T for music listening, SEED for movie watching, and VIP_Y_HYT for meditating. EEG activity is transformed into deep representation using a convolutional neural network comprising three different types of 2D convolutions: Temporal, Spatial, and Separable, to capture dependencies and extract high-level features from the data. The Depthwise Convolution function is responsible for learning spatial filters within each temporal convolution, and combining these spatial filters across all temporal bands optimally is learned by the Separable Convolutions. EEGNet and EEGNet-SSVEP are specially designed for EEG Signal Processing and Classification, and the DeepConvNet has incorporated more convolution layers. Our finding demonstrates that increasing the number of layers in the Network provided a higher accuracy of 99.94% using DeepConvNet. In contrast, the accuracy of EEGNet and EEGNet-SSVEP resulted in 85.63% and 75.76%, respectively., Design Aesthetics

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training., The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/No changes have been made to the publication.

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training., The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/No changes have been made to the publication.

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training., The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/No changes have been made to the publication.

We provide a user-friendly, flexible, and lightweight open-source HCI toolkit (github.com/QButterfly) that allows non-tech-savvy researchers to conduct online user interaction studies using the widespread Qualtrics and LimeSurvey platforms. These platforms already provide rich functionality (e.g., for experiments or usability tests) and therefore lend themselves to an extension to display stimulus web pages and record clickstreams. The toolkit consists of a survey template with embedded JavaScript, a JavaScript library embedded in the HTML web pages, and scripts to analyze the collected data. No special programming skills are required to set up a study or match survey data and user interaction data after data collection. We empirically validated the software in a laboratory and a field study. We conclude that this extension, even in its preliminary version, has the potential to make online user interaction studies (e.g., with crowdsourced participants) accessible to a broader range of researchers.

Photography in augmented reality can he challenging due to the restrictions of pre-defined settings. However, adjustable photography settings and real-time previews are significant for AR photographic creation as creators must adjust multiple camera properties to present unique visual effects. In this work, we designed an AR camera (ARCam) with various adjustable properties to give users a high degree of freedom for photographic art creation in real-time preview.

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example-based chart retrieval methods are built upon non-decoupled and low-level visual features that are hard to interpret, while definition-based ones are constrained to pre-defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What-You-Think-Is-What-You-Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language-image pre-training (CLIP) model to achieve zero-shot classification as well as multi-modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.

Live coding—synchronously writing software in front of students for the purpose of teaching—can be an effective method for engaging students and instilling practical programming skills. However, not all live coding sessions are effective and not all instructors are successful in this challenging task. We present results from an interview study involving university instructors,teaching assistants, and students identifying both barriers and benefits of live coding. We also designed and collected participant feedback on a prototype live coding tool to better facilitate learner engagement with the live coding pedagogical practice. Finally, we use this feedback to propose guidelines for how to design tools to support effective live coding in the classroom. This work advances our understanding of the benefits and challenges of live coding in university computer science instruction and highlights potential future work on the design of tools to better support this productive instructional practice.

Research on the nature and origins of human motivation has addressed the role of rewards in learning and behaviour. Gamification finds its raison d'être in being able to leverage motivational theories, to foster motivation in users through the use of game elements. One of the main criticisms moved to the use of gamification for learning purposes is related to the one-size-fits-all approach that tends to characterize many gamified applications. In this paper we explore the possibilities that can arise from the convergence of Self-Determination Theory principles and machine learning, to improve the efficacy of gamification reward systems.

The metaverse promises large-scale and persistent environments for users to share experiences at the intersection between physical and digital. Augmented reality (AR) is one of the primary technologies that supports blending digital content with physical environments. However, most AR applications are currently single-user and present significant constraints in terms of time and location. Scaling up the technology to large-sized environments and massively multi-user experiences thus represents one of the primary challenges to achieving such a vision. Due to these constraints, current proposals for the metaverse primarily consist of virtual worlds with little duality between physical and digital. This paper studies the feasibility of a large-scale and persistent AR experience shared among all visitors on our university campus. We define an integrated framework to enable the first AR campus metaverse by considering the nature of the various environments and the available sensing platforms. © 2022 IEEE.

In the use of Machine Learning systems, attaining the trust of those that are the end-users can often be difficult. Many of the current state-of-the-art systems operate as Black-Boxes. Errors produced by these Black-Box systems, without further explanation as to why these decisions were made, will deteriorate trust. This effect is especially strong when these erroneous decisions are generated with high confidence. This thesis presents both a data-driven as well as a human-in-the-Loop based methodology to characterize and mitigate high-confidence errors. We propose an Iterative Expert Session based methodology. By engaging domain experts through a series of interaction sessions, we aim to reduce the disconnect and knowledge gap between data scientists and domain experts, and to ultimately increase trust in the model. A practical approach was taken working in close connection with the practice of the data scientists of the ILT, helping them in improving their model and providing a direct contribution. We study the problem in the context of Road and Transportation law violations, by engaging inspectors (i.e., domain experts) in day-in-the-life and in-house interview sessions. A thorough analysis is performed of the most important features for data instances that were in error with a high degree of confidence. A method is presented that helps in characterizing these errors by predicting errors. We show that by careful removal of biased data features, proper data selection and by bridging the knowledge gap between domain experts and data scientists, we can improve the performance of the machine learning model. We show an increase of model Precision from 0.56800 with a baseline of 0.32968 to a Precision of 0.52077 with a baseline of 0.23473. Considering the baseline, this is an increase of 28.9% in Precision. We reduce biases existent in the data by reducing variables that predict on inspector practice. The magnitude of High Confidence Errors in the top 20% errors, Computer Science | Data Science and Technology

This article explains the new features of the Excite-O-Meter, an open-source tool that enables the collection of bodily data, real-time feature extraction, and post-session data visualization in any custom VR environment developed in Unity. Besides analyzing heart activity, the tool supports now multidimensional time series to study motion trajectories in VR. The paper presents the main functionalities and discusses the relevance of the tool for behavioral and psychophysiological research.

Ancient humans attribute the phenomenon of thunder and lightning to divine power. The power of Thor that can lift Thor's Hammer, the body not be hurt by thunder and lightning. It's not impossible for us to control thunder and lightning like Thor. The Digital Twins system of the robotic arm designed in this paper integrates the physical device of the robotic arm, the digital model of robotic arm, the body sense interaction, and the virtual-reality mapping module. It can digitally control the robotic arm. With this system, we can all lift Thor's hammer in the future.

Ocean waves provide a large amount of renewable energy, and Wave energy converter (WEC) can convert wave energy into electric energy. This paper proposes a visualization platform for wave power generation. The platform can monitor various indicators of wave power generation in real time, combined with Long Short-Term Memory (LSTM) neural network to predict wave power and electricity consumption. We make digital twins of a wave power plant in a computer, allowing users to remotely view the factory through VR glasses.

As a new type of functional material, magnetic fluid has both the fluidity of liquid and the magnetic properties of solid magnetic material. By controlling the magnets, one can simulate the effect of manipulating liquids like a sea emperor. This will provide new ideas for the multiverse of the metaverse. Not only that, magnetic fluids also have very important applications in astrophysics, controlled thermonuclear reactions and even the medical industry. Therefore, this paper hopes to provide a control idea for the future application of magnetic fluid by performing Digital Twins simulation of magnetic fluid.

Ocean waves provide a large amount of renewable energy, and Wave energy converter (WEC) can convert wave energy into electric en-ergy. This paper proposes a visualization platform for wave power generation. The platform can monitor various indicators of wave power generation in real time, combined with Long Short-Term Memory (LSTM) neural network to predict wave power and electric-ity consumption in real time and visualize monitoring data. The plat-form can intelligently allocate power generation equipment based on the power generation forecast data to achieve precise matching of power generation and power consumption, thereby improving overall power generation efficiency.

In this paper, we introduce the SSV360 dataset containing psy-chophysical and psychophysiological data gathered in a pilot study on the effect of standing and seated viewing of 360° videos on subjective quality assessment. The stimuli were shown to the participants on a head-mounted display for both viewing conditions. The ground truth contained in the SSV360 dataset may support, e.g., benchmarking of immersive media processing algorithms and developing of objective perceptual quality metrics. © 2022 IEEE.

Evaluating User Experiences early in a development process can save both time and money by pro-actively mapping out user needs and behavior patterns. However, since most well-known UX-evaluation methods applies during or after user interaction, due to the “second-order” design problem of experiences being a byproduct of interaction, there is a desire within HCI for more early-stage UX-evaluation methods that could be applied to concept stages as well. This paper investigates the experiential evaluation of a storyboard and narrative through the Anticipated eXperience Method (AxE) and discusses how it compares to a re-iterated high-fidelity prototype created in Figma. The process of this study is described out of the context it has been executed in, which is together with the company Scania AB at their cabin production facility in Oskarshamn, Sweden. The study explores what insights can be found from evaluating anticipated user experiences in early concept development and how these insights can apply towards further development of a touchpad user Interface. The underlying foundation of this study has followed the approach of a design-inclusive UX-research project, which heavily incorporates design activities into the process of conducting research. Thus, the storyboard, narrative and interactive Figma prototype have been created along the process and takes center stage in the investigation of experiential evaluation at early stages of interactive product development.

Nowadays, buying a product online is no longer about the product itself but the experience it offers. The planned thesis work aims to understand how to improve the user's shopping experience in the context of online shopping for the fashion industry. To enhance the shopping experience, retailers need to sell new services by leveraging technologies such as Virtual Reality (VR). The first part of the research investigates which shopping experience, between one developed for a desktop computer-Desktop Virtual Reality (DVR)-and one developed in Virtual Reality (VR), generates better results in terms of hedonic and utilitarian values, cognitive load, and user experience. Also, the lack of touch in online shopping is a crucial issue. The second part of the research concerns the implementation of pseudo-haptics feedback within the online shopping experience with VR. Pseudo-haptics can induce haptic sensations without requiring actual touch through the influence of other sensory modalities, such as vision. To this end, we aim to explore the feasibility of recreating the sensation of people's actual touch with fashion products and fabrics through a 'visualized touch' on an interface., Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Human Information Communication Design

Despite the advances made in Virtual Reality (VR) technology, the design of VR experiences lacks sufficient focus on accessibility and inclusion as the primary requirements. These are especially important for STEM education, where engaging in experiential activities is essential. This study was conducted to investigate accessibility considerations in the design and development of Immersive VR (IVR) learning spaces for wheelchair users. The specific research question is: How can we make a VR system easier to interact with for wheelchair users needing vertical movement? A user study with thirty (30) participants in three groups was conducted: Group A (the control group, non-wheelchair users) who used natural body movement to interact with the environment, Group B (verification group, non-wheelchair users) who used software controls for accessibility, and Group C (wheelchair users) who used the same software accessibility feature. The results indicate that the accessibility feature enabled wheelchair users to complete the tasks requiring raising or lowering of the body, with almost similar levels of completion rate and accuracy., Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., ChemE/Materials for Energy Conversion and Storage

Reflectance Imaging Spectroscopy (RIS) is a hyperspectral imaging technique used for investigating the molecular composition of materials. It can help identify pigments used in a painting, which are relevant information for art conservation and history. For every scanned pixel, a reflectance spectrum is obtained and domain experts look for pure representative spectra, called endmembers, which could indicate the presence of particular pigments. However, the identification of endmembers can be a lengthy process, which requires domain experts to manually select pixels and visually inspect multiple spectra in order to find accurate endmembers that belong to the historical context of an investigated painting. We propose an integrated interactive visual-analysis workflow, that combines dimensionality reduction and linked visualizations to identify and inspect endmembers. Here, we present initial results, obtained in collaboration with domain experts., Team Matthias Alfeld, Computer Graphics and Visualisation

Recently, with the development of stereo display and 3D graph-ics technology, virtual environments and applications are growing rapidly, e.g., video conferencing, and virtual reality (VR) applications. These are several E-commerce platforms that are designed for the transactions of VR applications. Consumers on these platforms can comment on a VR application after purchase. In this paper, we attempt to predict the helpfulness of these VR application reviews. To this end, we propose a topic-causal regression model that explores the influence of topic features in VR application reviews and nu-merical information on helpfulness. Specifically, we first apply the most classic topic model, Latent Dirichlet Allocation, to extract the topic signals in VR application reviews. Then, we construct a topic regression for causal inference. We perform extensive experiments on a real-world dataset collected from Oculus. The experimental results demonstrate that our model can estimate regression weights for topic factors and analyze their influence on the helpfulness of VR application reviews. © 2022 IEEE.

Cybersickness has been one of the main impediments to the widespread adoption of Virtual Reality for decades. It has been argued that several factors can influence the occurrence of cybersickness, such as technical factors, interaction design, but also users’ demographics and their perceived presence. Yet, previous studies had comparably small sample sizes and demographically homogeneous samples; comparisons across studies (e.g., regarding demographic factors) are challenging due to the large variation in the studied virtual environments. In this paper, we address these limitations and report the results of a lab-in-the-field experiment on cybersickness with a large and heterogeneous sample of N = 837 participants who navigated and interacted inside a virtual environment (ages 18–80, M = 29.34, SD = 9.50, 431 males, 400 females, 6 non-binaries and other). We found that female participants and participants with lower VR experience were more susceptible to experiencing higher levels of cybersickness. Participants’ cybersickness levels increased with the time spent in VR and with the distance traversed in the virtual world up to a point, above which reported levels declined. We also found a link between higher levels of cybersickness and reduced head motion, as well as between lower levels of cybersickness and more head motion, which led them to explore more of the virtual environment. In contrast to past studies, we did not find any evidence suggesting an effect of age on cybersickness, nor a negative correlation between presence and cybersickness. Based on our results, we derived a model that achieves a mean classification accuracy of 67.1% for two levels of cybersickness using demographic, user experience, and behavioral data in VR.