Search

Absolute Pose Regressors (APRs) directly estimate camera poses from monocular images, but their accuracy is unstable for different queries. Uncertainty-aware APRs provide uncertainty information on the estimated pose, alleviating the impact of these unreliable predictions. However, existing uncertainty modelling techniques are often coupled with a specific APR architecture, resulting in suboptimal performance compared to state-of-the-art (SOTA) APR methods. This work introduces a novel APR-agnostic framework, HR-APR, that formulates uncertainty estimation as cosine similarity estimation between the query and database features. It does not rely on or affect APR network architecture, which is flexible and computationally efficient. In addition, we take advantage of the uncertainty for pose refinement to enhance the performance of APR. The extensive experiments demonstrate the effectiveness of our framework, reducing 27.4% and 15.2% of computational overhead on the 7Scenes and Cambridge Landmarks datasets while maintaining the SOTA accuracy in single-image APRs.

Pervasive Augmented Reality (AR) requires accurate pose registration of the device in real-time at a neighbourhood-to-city scale. At such a scale, most pose registration techniques suffer from exponential computational and storage costs and a significant data collection burden. This paper introduces AnchorLoc, a framework that relies on visual anchors (stable and highly recognisable visual elements in a scene) to perform fast and accurate pose registration. Anchorloc automatically identifies these anchors from large image sequences to optimise the search space in later image retrieval and pose registration. As such, it significantly improves the computational efficiency of existing hierarchical localisation pipelines without compromising accuracy. We collect a large-scale localisation dataset consisting of image sequences and 3D reconstruction of a university campus. AnchorLoc reduces localisation runtime by 83% on our campus dataset and 69% on the Cambridge Landmarks dataset without significantly increasing mean pose estimation errors. It is also more accurate and faster than SLD, a localisation algorithm that takes a comparable approach at the keypoint level. This work informs the development of more efficient pervasive AR applications that rely on both absolute and relative camera pose tracking on image sequences. © 2024 IEEE.

Accurate camera relocalisation is a fundamental technology for extended reality (XR), facilitating the seamless integration and persistence of digital content within the real world. Benchmark datasets that measure camera pose accuracy have driven progress in visual re-localisation research. Despite notable progress in this field, there is a limited availability of datasets incorporating Visual Inertial Odometry (VIO) data from typical mobile AR frameworks such as ARKit or ARCore. This paper presents a new dataset, MARViN, comprising diverse indoor and outdoor scenes captured using heterogeneous mobile consumer devices. The dataset includes camera images, ARCore or ARKit VIO data, and raw sensor data for several mobile devices, together with the corresponding ground-truth poses. MARViN allows us to demonstrate the capability of ARKit and ARCore to provide relative pose estimates that closely approximate ground truth within a short timeframe. Subsequently, we evaluate the performance of mobile VIO data in enhancing absolute pose estimations in both desktop simulation and user study. MARViN is available at https://github.com/XRIM-Lab/MarViN.

Recent years have seen significant improvement in absolute camera pose estimation, paving the way for pervasive markerless Augmented Reality (AR). However, accurate absolute pose estimation techniques are computation- and storage-heavy, requiring computation offloading. As such, AR systems rely on visual-inertial odometry (VIO) to track the device’s relative pose between requests to the server. However, VIO suffers from drift, requiring frequent absolute repositioning. This paper introduces MobileARLoc, a new framework for on-device large-scale markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. Absolute pose regressors (APRs) provide fast on-device pose estimation at the cost of reduced accuracy. To address APR accuracy and reduce VIO drift, MobileARLoc creates a feedback loop where VIO pose estimations refine the APR predictions. The VIO system identifies reliable predictions of APR, which are then used to compensate for the VIO drift. We comprehensively evaluate MobileARLoc through dataset simulations. MobileARLoc halves the error compared to the underlying APR and achieves fast (80ms) on-device inference speed.

Recent advances in extended reality (XR) technologies have enabled new and increasingly realistic empathy tools and experiences. In XR, all interactions take place in different spatial contexts, all with different features, affordances, and constraints. We present a systematic literature survey of recent work on empathy in XR. As a result, we contribute a research roadmap with three future opportunities and six open questions in XR-enabled empathy research across both physical and virtual spaces. © 2023 Copyright held by the owner/author(s).

VR technology has brought new prospects and energy to art museums, prompting the advancement of new technologies that can tackle future challenges while catering to the demands of the metaverse era. Today’s digital archiving and guided tours, which rely on VR and more portable information retrieval, provide optimism for museum development. VR technology overcomes the limitations of traditional museums constrained by space and time as digital archiving gradually becomes a reality. This study highlights the benefits of metaverse-driven online archiving from the users’ perspective, followed by a review of recent examples of using VR for media art archiving. We emphasize the importance of further research into unexplored emerging technologies in media art archiving and propose that VR technologies in the metaverse are essential enablers for online archiving purposes. Our work aims to help art researchers, archivists, professionals, and artists to comprehend the challenges and future needs of archiving media art in the developing metaverse. Furthermore, we discuss present cases of employing VR for media art preservation after outlining the advantages of metaverse-driven online archiving from the users’ viewpoint. We stress the need for additional studies on new technologies in media art preservation. Moreover, metaverse VR technologies play a crucial role in achieving online archiving goals. This research contributes to a better understanding of the challenges and future requirements of archiving media art in the evolving metaverse for art researchers, archivists, and artists.

The Metaverse connects our physical reality with virtual worlds. Social VR platforms facilitate the creation of such virtual worlds, enabling activities such as interactive teaching, conferences, and community gatherings. These activities can be performed in mixed mode, with some participants physically present in the same location. In this paper, we evaluate the feasibility of such mixed-mode events by studying three leading social VR platforms. We uncover the network traffic patterns generated by these platforms, which affect the user experience when multiple users share the same network. We explore the traffic patterns to show that model loading creates a significant overhead and impacts user quality of experience. When the number of simultaneously connected users increases, some operations lead to network congestion that degrades or even interrupts service for most users. From these observations, we derive concrete action points to develop social VR platforms that enable mixed-mode events over the same network.

This chapter focuses on discussing the evolution of the Metaverse as well as the difficulties that have been encountered. It provides a concise overview of the evolution of cyberspace as well as the importance of technological enablers. Edge computing can reduce the user-experienced latency when compared with cloud offloading. The chapter emphasizes a number of essential challenges, both from a technical and an ecosystemic point of view, that are necessary for the construction and maintenance of the Metaverse. It looks at bottom-up approach to discuss the three most important aspects, which have been inherited from the viewpoints of ubiquitous computing in particular: the network and mobile edge, system interoperability, and user-centric immersive design, which are representing the network, the system, and human users. All the immersive technologies that will deliver entry points to Metaverse via multiple types of devices will have access to potentially private users' data. © 2024 The Institute of Electrical and Electronics Engineers, Inc. All rights reserved.

The emergence of the Metaverse enables the creation of alternative spaces at the intersection between digital and physical through the replication of physical events and objects within physical-digital twins. In this article, we apply such twins to connected vehicles within a Traffic Metaverse as an intermediate platform for the shared perception of the road environment. The platform builds a real-time digital copy of the road conditions to serve vehicular applications on a city scale through ubiquitous sensing, reliable and low-latency communications, artificial intelligence, and extended reality. This article focuses on collaboratively building virtual 3D maps of road networks that provide road users with a pervasive view of the road conditions to increase situational awareness. Through vehicle-to-everything (V2X) cooperative perception, such a Metaverse platform expands the driver's visibility beyond the vehicle's line of sight. The platform also enables virtual remote access to monitor and forecast traffic ahead of space and time. © 2017 IEEE.

While the academic community tries to define and experiment with the metaverse, businesses and institutions seek to build their representation in the metaverse. Many educational institutions build meta-campuses and move online classes into virtual environments beyond simple videoconferencing. This paper describes our experience building a university metaverse, highlighting the technical, human, and organisational challenges we have encountered through two major events. Considering the issue of real-time communication and scalability in the web-based metaverse, we also present an analysis of video streaming in virtual reality (VR) platforms: Meta's Workroom, Spatial, and Mozilla Hubs.

This proposal describes the design and demonstration of the Unlogical Instrument. This textile-based gesture-controlled sound interface invites the audience to generate sound by interacting with the textile using different gestures, such as poking, flicking, and patting. Unlogical is a statement about breaking the dogmatic approach of commonly used musical instruments. It brings resilience to the form of musical expression and interaction for the audience. It introduces an interactive mechanism mediated by intuitive human gestures and the curiosity to explore tangible material. Unlogical Instrument locates the focus on the textile. This artwork investigates the relationship between the gestures and the sound perception that originates from the textile through a material-driven approach. Inkjet printing is applied to transfer the original textile into a sensorial interface. The audience will listen to the sound while interacting with the textile surface, exploring and understanding this instrument through free-form playing.

With the advancement of virtual reality (VR) technology, virtual displays have become integral to how museums, galleries, and other tourist destinations present their collections to the public. However, the current lack of immersion in virtual reality displays limits the user's ability to experience and appreciate its aesthetics. This paper presents a case study of a creative approach taken by a tourist attraction venue in developing a physical network system that allows visitors to enhance VR's aesthetic aspects based on environmental parameters gathered by external sensors. Our system was collaboratively developed through interviews and sessions with twelve stakeholder groups interested in art and exhibitions. This paper demonstrates how our technological advancements in interaction, immersion and visual attractiveness surpass those of earlier virtual display generations. Through multimodal interaction, we aim to encourage innovation on the Web and create more visually appealing and engaging virtual displays. It is hoped that the greater online art community will gain fresh insight into how people interact with virtual worlds as a result of this work. © 2023 ACM.

Wearable computers have seen a recent resurgence in interest and popularity in which augmented reality (AR) smartglasses are poised to influence the way we complete our work and daily tasks. Nowadays, industrial applications of these smartglasses are focused on interior designs, remote collaborations, as well as e-commerce. Under five key constraints on AR smartglasses such as miniature touch interface, small-screen real estate, user mobility, limited computational resource, and short battery life, existing user interaction paradigms designed for desktop computers and smartphones are obsolete and incompatible with the scenarios of AR smartglasses. The cumbersome and difficult interaction with the AR smartglasses becomes a hurdle to their wider industrial applications. Thus, an unmet demand for designing interaction techniques on AR smartglasses is undoubtedly critical. In this chapter, we present three interaction techniques, namely, TiPoint, HIBEY, and TOFI, in order to enhance object manipulation and text entry in the constrained environment of AR smartglasses. These techniques are devised in a way that leverage on advantageous features of human body and experiences such as the dexterity of fingertip, lexicographical order ingrained in our memory, proprioception, as well as opposable thumbs. We thoroughly address the key constraints on AR smartglasses and explore different modalities with various hardware and peripherals. © 2023, Springer Nature Switzerland AG.

Mobile augmented reality (MAR) applications are gaining popularity due to the wide adoption of mobile and especially wearable devices such as smartglasses. These devices often strike a compromise between mobility, energy efficiency, and performance. On the other hand, MAR applications rely on computationally intensive computer vision algorithms with extreme latency requirements. Cyber-foraging allows resource-constrained devices to leverage the computing power of nearby machines, and has often been proposed as a solution for increasing the computing capabilities of constrained devices. However, this process introduces new constraints in the application, especially in terms of latency and bandwidth. MAR applications are so demanding that current network infrastructures are barely ready for such traffic. Such resource-hungry applications may rapidly saturate future wireless networks such as 5G. As such, developing cyber-foraging solutions for MAR applications requires a delicate balance in network resource usage to ensure seamless user experience. In order to identify the opportunities for further improving the Quality of Experience (QoE) for mobile AR, we break down the end-to-end latency of the pipeline for typical MAR applications and pinpoint the dominating components in the critical path, from the physical transmission to the application. We then derive a set of actions to enhance the possibilities of cyber-foraging, taking into account the specificities of future networking technologies. © 2023, Springer Nature Switzerland AG.

Driven by the recent improvements in device and networks capabilities, Extended Reality (XR) is becoming more pervasive; industry and academia alike envision ambitious projects such as the metaverse. However, XR is still limited by the current architecture of mobile systems. This article makes the case for an XR-specific operating system (XROS). An XROS integrates hardware-support, computer vision algorithms, and XR-specific networking as the primitives supporting XR technology. These primitives represent the physical-digital world as a single shared resource among applications. Such an XROS allows for the development of coherent and system-wide interaction and display methods, systematic privacy preservation on sensor data, and performance improvement while simplifying application development. IEEE

We present VR PreM+, an innovative VR system designed to enhance web exploration beyond traditional computer screens. Unlike static 2D displays, VR PreM+ leverages 3D environments to create an immersive pre-learning experience. Using keyword-based information retrieval allows users to manage and connect various content sources in a dynamic 3D space, improving communication and data comparison. We conducted preliminary and user studies that demonstrated efficient information retrieval, increased user engagement, and a greater sense of presence. These findings yielded three design guidelines for future VR information systems: display, interaction, and user-centric design. VR PreM+ bridges the gap between traditional web browsing and immersive VR, offering an interactive and comprehensive approach to information acquisition. It holds promise for research, education, and beyond.

The metaverse aims to blur the boundary between the physical world and digital content. To achieve this goal, the metaverse relies heavily on extended reality (XR), the Internet of Things, and communication technologies. Concurrently, connected vehicles and intelligent transportation systems (ITSs) are envisioned as the future paradigm of driving and becoming reality thanks to increasingly powerful onboard vehicular processing capacity and advanced vehicle-to-everything networking technologies. © 2005-2012 IEEE.

This paper presents "AI Nüshu," an emerging language system inspired by Nüshu (women's scripts), the unique language created and used exclusively by ancient Chinese women who were illiterate under a patriarchy society. Through an interactive art installation, two artificial intelligent (AI) agents continuously observe their environment and communicate with each other, developing a writing system that encodes Chinese. In this system, two AI agents observe the environment through cameras, record the unconscious behaviors of the audience, and generate summaries of their observations through visual recognition. Subsequently, the agent associates the corresponding original Nüshu poetry lines and generates new poetry text through a Language Model (LLM), representing its reflection. To develop their language, they continuously switch roles between the speaker and listener, constantly communicating their reflections, and encrypting a word in the poetry line with their self-created AI Nüshu character, allowing the other to guess and learn. Gradually, they reach a consensus on AI Nüshu, forming a unique "AI Nüshu Dictionary" for machines. This language, algorithmically combined into corresponding characters, has components derived from Nüshu, similar to Chinese characters and traditional textile patterns. Thus, like ancient women, the two agents gradually developed their Chinese writing system, corresponding one-to-one with Chinese characters. In contrast, humans, as the authority of the language system, became an object observed, interpreted, and inspired by machines to stimulate non-human language. This is the first media art project to interpret Nüshu from a computational linguistics perspective, infusing AI and art research with non-English natural language processing, Chinese cultural heritage, and a feminist viewpoint. This encourages the creation of more non-English, linguistically-oriented artworks for diverse cultures. We simulate communication in sisterhood through

This paper presents “AI Nüshu," an emerging language system inspired by Nüshu (women’s scripts), the unique language created and used exclusively by ancient Chinese women who were thought to be illiterate under a patriarchal society. In this interactive installation, two artificial intelligence (AI) agents are trained in the Chinese dictionary and the Nüshu corpus. By continually observing their environment and communicating, these agents collaborate towards creating a standard writing system to encode Chinese. It offers an artistic interpretation of the creation of a non-western script from a computational linguistics perspective, integrating AI technology with Chinese cultural heritage and a feminist viewpoint.

WallStreetBets (WSB), a Reddit community, impacted stock markets during the 2021 GameStop Short Squeeze. We examine the content and user properties that influence engagement in WSB. Despite WSB's association with emojis and informal terms, engagement among community members depends on more than surface-level factors. Although emojis are commonly used, they are not as effective at fostering interactions among users. Community members engage more with posts that have longer and topic-specific text. Simply producing a high volume of posts is not enough to attract an audience. Consistent topical focus, reciprocal interactions, and previous authorship of catalyst posts influence engagement. WSB posts, regardless of length, generally remain relevant to the community's theme of stock trading. Our findings provide insights into WSB engagement patterns and can be useful for downstream research, such as financial predictive tasks using WSB data.

Interaction design for Augmented Reality (AR) is gaining attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015-2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of I luman-City Interaction and related technological approaches, followed by reviewing the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a crucial enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction.

Mobile cloud gaming enables high-end games on constrained devices by streaming the game content from powerful servers through mobile networks. Mobile networks suffer from highly variable bandwidth, latency, and losses that affect the gaming experience. This paper introduces , an end-to-end cloud gaming framework to minimize the impact of network conditions on the user experience. relies on an end-to-end distortion model adapting the video source rate and the amount of frame-level redundancy based on the measured network conditions. As a result, it minimizes the motion-to-photon (MTP) latency while protecting the frames from losses. We fully implement and evaluate its performance against the state-of-the-art techniques and latest research in real-time mobile cloud gaming transmission on a physical testbed over emulated and real wireless networks. consistently balances MTP latency (<140 ms) and visual quality (>31dB) even in highly variable environments. A user experiment confirms that maximizes the user experience with high perceived video quality, playability, and low user load.

The General Data Protection Regulation (GDPR) presents directives to give data subjects control over their personal data. These directives impose data-collecting and processing organizations to take concrete actions for privacy preservation of users and non-users alike. Significant challenges arise when applying these directives to mobile and ubiquitous computing. Mobile and ubiquitous computing aim for computer use to be as transparent and seamless as possible. Inconspicuous devices continually sense their environment, often without the data subject's knowledge. This context significantly complicates the implementation of core GDPR directives, such as informing the user and collecting consent. In this paper, we challenge the mobile computing research community on how to address such issues in practical implementations that combine the philosophy of mobile and ubiquitous computing with often constraining privacy-regulations. © 2022 ACM.

The metaverse promises large-scale and persistent environments for users to share experiences at the intersection between physical and digital. Augmented reality (AR) is one of the primary technologies that supports blending digital content with physical environments. However, most AR applications are currently single-user and present significant constraints in terms of time and location. Scaling up the technology to large-sized environments and massively multi-user experiences thus represents one of the primary challenges to achieving such a vision. Due to these constraints, current proposals for the metaverse primarily consist of virtual worlds with little duality between physical and digital. This paper studies the feasibility of a large-scale and persistent AR experience shared among all visitors on our university campus. We define an integrated framework to enable the first AR campus metaverse by considering the nature of the various environments and the available sensing platforms. © 2022 IEEE.

In this paper, we highlight the use of Instagram for social activism, taking 2019 Hong Kong protests as a case study. Instagram focuses on image content and provides users with few features to share or repost, limiting information propagation. Nevertheless, users who are politically active offline also share their activism on Instagram. We first evaluate the effect of protests on social media activity for protesters and non-protesters over two significant protests. Protesters’ exposure to protest-related posts is much higher than non-protesters, and their network activity follows the protest schedule. They are also much more active on posts related to the protest that they participate in than the other protest. We then analyze the images posted by the users. Users predominantly use symbols related to protests and share personal thoughts on its primary actors. Users primarily share content to raise their network’s awareness, and the content choice is directly affected by Instagram’s intrinsic interaction modalities.

Connected vehicles, whether equipped with advanced driver-assistance systems or fully autonomous, require human driver supervision and are currently constrained to visual information in their line-of-sight. A cooperative perception system among vehicles increases their situational awareness by extending their perception range. Existing solutions focus on improving perspective transformation and fast information collection. However, such solutions fail to filter out large amounts of less relevant data and thus impose significant network and computation load. Moreover, presenting all this less relevant data can overwhelm the driver and thus actually hinder them. To address such issues, we present Augmented Informative Cooperative Perception (AICP), the first fast-filtering system which optimizes the informativeness of shared data at vehicles to improve the fused presentation. To this end, an informativeness maximization problem is presented for vehicles to select a subset of data to display to their drivers. Specifically, we propose (i) a dedicated system design with custom data structure and lightweight routing protocol for convenient data encapsulation, fast interpretation and transmission, and (ii) a comprehensive problem formulation and efficient fitness-based sorting algorithm to select the most valuable data to display at the application layer. We implement a proof-of-concept prototype of AICP with a bandwidth-hungry, latency-constrained real-life augmented reality application. The prototype adds only 12.6 milliseconds of latency to a current informativeness-unaware system. Next, we test the networking performance of AICP at scale and show that AICP effectively filters out less relevant packets and decreases the channel busy time.

The authors present the visual display installation as the basis for creating the artwork series 'Metanalysis'. The work explores the visual algorithmic representation of traditional Chinese ink painting and provides a video installation designed to showcase this visualization project. Exploring these visual representations is an attempt to understand and modernize expressions in the developing digital age visually. These visual displays are rendered in real-time and produced one after another in a continuous overlap and accumulation of black ink lines to reflect the specific random and malleable characteristics of the ink. They were eventually using mathematical algorithms to generate synthetic images that mimic the process of staining ink. The value of this paper lies in the fact that it proposes a new path for the expression of traditional Chinese ink painting and aims to bring the nonvitality classic art back to the stage through digital forms and to provide contemporary artists with new ideas to promote traditional culture. © 2022 IEEE.

Augmented Reality (AR) visits enhances the art exhibition experience by overlaying digital content. Although there has been significant interest in AR guides, few works leverage AR to bridge curatorial intent and audiences understanding. This paper focuses on integrating the curatorial intent within the AR overlays by developing the narrative layers established by the relationships between works. We develop a narrative system that identifies and links the primary art pieces of the exhibition within a digital story consistent with the curator's perspective. The system is applied to a physical exhibition composed of seven art pieces. We evaluate the impact of AR overlays through two user experiments, conducted on art professionals and general audience, respectively. Both groups considered that the AR tour system improved interactivity, self-reported learning, and user satisfaction significantly (> 4/5). Besides, visitors found the system easy to get to do what they want to (4.7/5), and would use it for future visits (4.6/5). This study raises essential design considerations towards designing integrated AR museum guides that combine the perspective of artists and curators towards a better visiting experience. © 2022 ACM.

Tai Chi is an essential concept of Chinese philosophy which refers to the universe's most primitive state of order. Tai Chi Chuan is a martial art and meditative practice that incorporates the Tai Chi philosophy. With the advent of the digital media age, this traditional martial art is falling into disuse, and most contemporary youths are losing interest. "Meditation in Motion" is an interactive media art installation inspired by the Tai Chi Chuan forms. It aims to convey the central concepts of balance, narrative, and universe of Tai Chi Chuan by breaking down its movements into overlapping circles, building a visual representation of the energy flows in the body.

There has been a significant expansion in the use of online social networks (OSNs) to support people experiencing mental health issues. This paper studies the role of Instagram influencers who specialize in coaching people with mental health issues. Using a dataset of 97k posts, we characterize such users' linguistic and behavioural features. We explore how these observations impact audience engagement (as measured by likes). We show that the support provided by these accounts varies based on their self-declared professional identities. For instance, Instagram accounts that declare themselves as Authors offer less support than accounts that label themselves as a Coach. We show that increasing information support in general communication positively affects user engagement. However, the effect of vocabulary on engagement is not consistent across the Instagram account types. Our findings shed light on this understudied topic and guide how mental health practitioners can improve outreach. © 2022 IEEE.

User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behavior. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We present a continuum theory and propose a framework combining a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behavior data into generative NFT units, as

During the COVID-19 pandemic, most countries have experienced some form of remote education through video conferencing software platforms. However, these software platforms fail to reduce immersion and replicate the classroom experience. The currently emerging Metaverse addresses many of such limitations by offering blended physical-digital environments. This paper aims to assess how the Metaverse can support and improve e-learning. We first survey the latest applications of blended environments in education and highlight the primary challenges and opportunities. Accordingly, we derive our proposal for a virtual-physical blended classroom configuration that brings students and teachers into a shared educational Metaverse. We focus on the system architecture of the Metaverse classroom to achieve real-time synchronization of a large number of participants and activities across physical (mixed reality classrooms) and virtual (remote VR platform) learning spaces. Our proposal attempts to transform the traditional physical classroom into virtual-physical cyberspace as a new social network of learners and educators connected at an unprecedented scale. © 2022 IEEE.

Human habitation across multiple planets requires communication and social connection between planets. When the infrastructure of a deep space network becomes mature, immersive cyberspace, known as the Metaverse, can exchange diversified user data and host multitudinous virtual worlds. Nevertheless, such immersive cyberspace unavoidably encounters latency in minutes, and thus operates in a turn-taking manner. This Blue Sky paper illustrates a vision of an interplanetary Metaverse that connects Earthian and Martian users in a turn-based Metaverse. Accordingly, we briefly discuss several grand challenges to catalyze research initiatives for the ‘Digital Big Bang’ on Mars.

The computational capabilities of recent mobile devices enable the processing of natural features for Augmented Reality (AR), but the scalability is still limited by the devices' computation power and available resources. In this paper, we propose EdgeXAR, a mobile AR framework that utilizes the advantages of edge computing through task offloading to support flexible camera-based AR interaction. We propose a hybrid tracking system for mobile devices that provides lightweight tracking with 6 Degrees of Freedom and hides the offloading latency from users' perception. A practical, reliable and unreliable communication mechanism is used to achieve fast response and consistency of crucial information. We also propose a multi-object image retrieval pipeline that executes fast and accurate image recognition tasks on the cloud and edge servers. Extensive experiments are carried out to evaluate the performance of EdgeXAR by building mobile AR apps upon it. Regarding the Quality of Experience (QoE), the mobile AR apps powered by EdgeXAR framework run on average at the speed of 30 frames per second with precise tracking of only 1-2 pixel errors and accurate image recognition of at least 97% accuracy. As compared to Vuforia, one of the leading commercial AR frameworks, EdgeXAR transmits 87% less data while providing a stable 30FPS performance and reducing the offloading latency by 50 to 70% depending on the transmission medium. Our work facilitates the large-scale deployment of AR as the next generation of ubiquitous interfaces.

Social media is often used to disseminate information during crises, including wars, natural disasters and pandemics. This paper discusses the challenges faced during crisis situations, which social media can both contribute to and ameliorate. We discuss the role that information polarisation plays in exacerbating problems. We then discuss how certain mal-actors exploit these divides. We conclude by detailing future avenues of work that can help mitigate these issues.

WallStreetBets (WSB) is a Reddit community that primarily discusses high-risk options and stock trading. In January 2021, it attracted worldwide attention as one of the epicentres of a significant short squeeze on US markets. Following this event, the number of users and their activity increased exponentially. In this paper, we study the changes caused in the WSB community by such an increase in activity. We perform a comparative analysis between long-term users and newcomers and examine their respective writing styles, topics, and susceptibility to community feedback. We report a significant difference in the post length and the number of emojis between the regular and new users joining WSB. Newer users’ activity also closely follows the affected companies’ stock prices. Finally, although community feedback affects the choices of topics for all users, new users are less prone to select their subsequent message topics based on past community feedback.

The COVID-19 pandemic led governments to rely on the versatility of social media to communicate with their citizens. This paper analyzes the Facebook communication of political leaders and health departments from 17 countries during the COVID-19 pandemic. We evaluate the citizen's response under the frameworks of media richness and user engagement. We note that governments and leaders communicate primarily through richer media (photos and videos), despite a negative correlation between media richness and user engagement. Plain-text messages posted by country leaders attract the most engagement, while their COVID-19 communication tends to generate lower engagement. On the other hand, health departments' pages experienced a sharp increase in engagement around COVID-19 communication as citizens sought information during the pandemic. Finally, topical analysis shows that Discussions and Co-Use of COVID-19 with other topics can boost citizens' engagement. Our comparative analysis shows practical implications for social media users and social media designers. Our findings can help governments and organizations design effective social media communication during crises.

Recent advances in information technology have revolutionized the automotive industry, paving the way for next-generation smart vehicular mobility. Specifically, vehicles, roadside units, and other road users can collaborate to deliver novel services and applications that leverage, for example, big vehicular data and machine learning. Relatedly, fifth-generation cellular networks (5G) are being developed and deployed for low-latency, high-reliability, and high bandwidth communications. While 5G adjacent technologies such as edge computing allow for data offloading and computation at the edge of the network thus ensuring even lower latency and context-awareness. Overall, these developments provide a rich ecosystem for the evolution of vehicular applications, communications, and computing. Therefore in this work, we aim at providing a comprehensive overview of the state of research on vehicular computing in the emerging age of 5G and big data. In particular, this paper highlights several vehicular applications, investigates their requirements, details the enabling communication technologies and computing paradigms, and studies data analytics pipelines and the integration of these enabling technologies in response to application requirements. © 2013 IEEE.

We quantify and derive a general model for the collaboration stability of human mobility and demonstrate its importance for networking applications. Our results demonstrate that collaboration opportunities are highly dependent on the context where they take place, with diurnal patterns and spatial characteristics being particularly important. © 1970-2012 IEEE.

Urban events are the sets of real-world occurrences in urban space associated with specific topics. Discovering, understanding, and forecasting various urban events are essential to facilitate smart-city applications and benefit urban life. In recent years, mobile devices and sensors widely located in cities have collected large amounts of data produced in urban space, which form a large-scale, cross-domain, and multi-view data ecosystem. The collected data, termed urban big data, provide an unprecedented opportunity to discover and analyze urban events. In this thesis, we develop data-driven methodologies for urban events mining. We present four of our works on urban anomaly detection, urban location representation learning, road network traffic prediction, and urban events prediction, respectively. First, we introduce an urban dynamic decomposition method to detect urban anomalies. We propose a neural network model to estimate normal urban dynamics using spatiotemporal features and detect abnormal events via residual analysis. We validate the effectiveness of the proposed method with both synthetic and real-world event datasets. Regions are the primary spatial units for urban event analysis. The second work focuses on learning an embedding space from urban big data for urban regions. We propose to construct multi-view relations among urban regions and learn urban region embeddings through a multi-view joint learning model. Experiments on real-world datasets prove the effectiveness of the learned embeddings on downstream applications, including predicting long-term regional event statistics such as crime rates. Traffic prediction is a crucial task for congestion events diagnosis and control. In the third work, we propose a traffic prediction model called Adaptive Spatiotemporal Convolution Network, which models complicated spatiotemporal traffic dependencies adaptively with awareness of the real-time traffic conditions. Experiments on real-world traffic datasets fr

The modern urban landscape is becoming increasingly connected, from distributed vehicular safety systems to in-vehicle centralized entertainment. These technologies face network challenges related to the high mobility of vehicles and people. Such mobility raises new challenges. On-board detection of unsafe driving activities lacks a holistic view of the road situation while cloud-offloaded detection faces scalability and context-awareness issues. In terms of in-vehicle entertainment, the connection to remote gaming and VR servers encounters variable bandwidth, latency, and packet losses that affect the gaming and VR experience. This thesis presents four research contributions to provide reliable real-time content streaming for driving awareness (in the physical world), mobile cloud gaming (in the virtual world), and a reality-check of the Metaverse between physical and virtual worlds. The first contribution is CAD3, a distributed collaborative architecture for road-aware and driver-aware anomaly driving detection and real-time warning dissemination. CAD3 exploits the pervasive deployment of roadside units, and combines collaborative and context-aware computation with low-latency communication to detect unsafe driving behaviors and warn the drivers of nearby vehicles in real-time. The second contribution is Nebula, an end-to-end cloud gaming architecture to minimize the impact of network conditions on the user experience. Nebula relies on a heuristic algorithm and end-to-end distortion model to dynamically adapt the video bitrate and redundancy based on the measured network conditions. The third contribution is MERA, an edge-assisted learning-based end-to-end cloud gaming architecture to adapt the video bitrate to the network constraints. MERA relies on the transition, state-to-action mapping, then rewarding with a multi-objective reward function to maximize the user QoE. The fourth contribution is Metaversity, a reality check of social VR platforms. It uncovers the

Speeding, slowing down, and sudden acceleration are the leading causes of fatal accidents on highways. Anomalous driving behavior detection can improve road safety by informing drivers who are in the vicinity of dangerous vehicles. However, detecting abnormal driving behavior at the city-scale in a centralized fashion results in considerable network and computation load, that would significantly restrict the scalability of the system. In this paper, we propose CAD3, a distributed collaborative system for road-aware and driver-aware anomaly driving detection. CAD3 considers a decentralized deployment of edge computation nodes on the roadside and combines collaborative and context-aware computation with low-latency communication to detect and inform nearby drivers of unsafe behaviors of other vehicles in real-time. Adjacent edge nodes collaborate to improve the detection of abnormal driving behavior at the city-scale. We evaluate CAD3 with a physical testbed implementation. We emulate realistic driving scenarios from a real driving data set of 3, 000 vehicles, 214, 000 trips, and 18 million trajectories of private cars in Shenzhen, China. At the microscopic (road) level, CAD3 significantly improves the accuracy of detection and lowers the number of potential accidents caused by false negatives up to four times and 24 times as compared to distributed standalone and centralized models, respectively. CAD3 can scale up to 256 vehicles connected to a single node while keeping the end-to-end latency under 50 ms and a required bandwidth below 5 mbps. At the mesoscopic (driver-trip) level, CAD3 performs stable and accurate detection over time, owing to local RSU interaction. With a dense deployment of edge nodes, CAD3 can scale up to the size of Shenzhen, a megalopolis of 12 million inhabitant with over 2 million concurrent vehicles at peak hours. © 2021 IEEE.

The sixth generation of mobile networks (6G) is expected to be deployed in the early 2030s. By this time, the density of autonomous Internet-connected machines is expected to explode up to hundreds of devices per cubic meter. These devices (1) generate voluminous multisensory data, (2) access sophisticated artificial intelligence-based services with high frequency, and (3) have widely diverse constraints in terms of latency, bandwidth, energy, and computation power. Such devices are not operated by humans, and communicate with each other and with remote servers located on the network core or edge. The wireless communications between these machines are called machine-type communications (MTC) and can either be between multiple machines that collectively collect and process multidimensional information or between machines that interact with services located on servers. Representative examples include autonomous driving, piloting crewless aerial vehicles, smart grid energy trading, and others. In this chapter, we define the following requirements for 6G, following the predicted density and heterogeneity of the autonomous connected device landscape: (1) ultra-dense wireless communication networks, (2) massive multi-access edge computing, (3) large-scale autonomous operation of devices with heterogeneous requirements and constraints. To address these requirements, 6G will enable a convergence of computing, energy, and communication for device- and application-aware communications. We discuss how 6G can achieve such convergence and highlight the future trends for MTC to ubiquitously integrate the computing landscape.

Many factors affect speech intelligibility in face-to-face conversations. These factors lead conversation participants to speak louder and more distinctively, exposing the content to potential eavesdroppers. To address these issues, we introduce Theophany, a privacy-preserving framework for augmenting speech. Theophany establishes ad-hoc social networks between conversation participants to exchange contextual information, improving speech intelligibility in real-time. At the core of Theophany, we develop the first privacy perception model that assesses the privacy risk of a face-to-face conversation based on its topic, location, and participants. This framework allows to develop any privacy-preserving application for face-to-face conversation. We implement the framework within a prototype system that augments the speaker's speech with real-life subtitles to overcome the loss of contextual cues brought by mask-wearing and social distancing during the COVID-19 pandemic. We evaluate Theophany through a user survey and a user study on 53 and 17 participants, respectively. Theophany's privacy predictions match the participants' privacy preferences with an accuracy of 71.26%. Users considered Theophany to be useful to protect their privacy (3.88/5), easy to use (4.71/5), and enjoyable to use (4.24/5). We also raise the question of demographic and individual differences in the design of privacy-preserving solutions.

Artificial intelligence shows promise for solving many practical societal problems in areas such as healthcare and transportation. However, the current mechanisms for AI model diffusion such as Github code repositories, academic project webpages, and commercial AI marketplaces have some limitations; for example, a lack of monetization methods, model traceability, and model auditabilty. In this work, we sketch guidelines for a new AI diffusion method based on a decentralized online marketplace. We consider the technical, economic, and regulatory aspects of such a marketplace including a discussion of solutions for problems in these areas. Finally, we include a comparative analysis of several current AI marketplaces that are already available or in development. We find that most of these marketplaces are centralized commercial marketplaces with relatively few models. CCBY

Mobile cloud gaming requires a very low end-to-end latency. Edge computing significantly reduces network latency. However, in mobility scenarios, the user will frequently move out of the edge server's coverage area, requiring frequent migration of the game instance. This paper presents Talaria, an in-engine content synchronisation solution for unnoticeable game instance migration between edge servers. Talaria creates a minimal instance with content immediately relevant to the game experience, allowing the client to switch servers in a minimal amount of time. The remaining content is then synchronised according to priority until the game's state is coherent between both instances. Our implementation of Talaria as a Unity engine plugin reduces the game's downtime by 61% compared to one-off server migration, with an average latency below 25 ms for the server migration, and 87 ms for the entire game synchronisation. © 2021 ACM.

The Internet of Vehicles (IoV) enables real-time data exchange among vehicles and roadside units and thus provides a promising solution to alleviate traffic jams in the urban area. Meanwhile, better traffic management via efficient traffic light control can benefit the IoV as well by enabling a better communication environment and decreasing the network load. As such, IoV and efficient traffic light control can formulate a virtuous cycle. Edge computing, an emerging technology to provide low-latency computation capabilities at the edge of the network, can further improve the performance of this cycle. However, while the collected information is valuable, an efficient solution for better utilization and faster feedback has yet to be developed for edge-empowered IoV. To this end, we propose a Decentralized Reinforcement Learning at the Edge for traffic light control in the IoV (DRLE). DRLE exploits the ubiquity of the IoV to accelerate traffic data collection and interpretation towards better traffic light control and congestion alleviation. Operating within the coverage of the edge servers, DRLE aggregates data from neighboring edge servers for city-scale traffic light control. DRLE decomposes the highly complex problem of large area control into a decentralized multi-agent problem. We prove its global optima with concrete mathematical reasoning and demonstrate its superiority over several state-of-the-art algorithms via extensive evaluations.