Search

With the rise of head-mounted displays (HMDs), immersive virtual reality (IVR) for second-language learning is gaining attention. However, current methods fail to fully exploit IVR’s potential owing to the use of abstract avatars and limited human perspectives in learning experiences. This study investigated IVR’s novel potential by using non-human avatars to understand complex concepts. We developed a system for learning English vocabulary through the actions of non-human avatars, offering a unique learning perspective. This paper presents an IVR vocabulary learning environment with a dragon avatar and compares word retention rates (immediate and one-week memory tests), subjective workload, and emotional changes with traditional methods. We also examined the vocabulary ranges that are teachable using this system by varying the number of avatars. The results showed that the proposed method significantly reduced forgotten English words after one week compared to traditional methods, indicating its effectiveness in the long term.

Subtitles play a crucial role in facilitating the understanding of visual content when watching films and television programs. In this study, we propose a method for presenting subtitles in a way that considers cognitive load when viewing video content in a non-native language. Subtitles are generally displayed at the bottom of the screen, which causes frequent eye focus switching between subtitles and video, increasing the cognitive load. In our proposed method, we focused on the position, display time, and amount of information contained in the subtitles to reduce the cognitive load and to avoid disturbing the viewer’s concentration. We conducted two experiments to investigate the effects of our proposed subtitle method on gaze distribution, comprehension, and cognitive load during English-language video viewing. Twelve non-native English-speaking subjects participated in the first experiment. The results show that participants’ gazes were more focused around the center of the screen when using our proposed subtitles compared to regular subtitles. Comprehension levels recorded using LightSub were similar, but slightly inferior to those recorded using regular subtitles. However, it was confirmed that most of the participants were viewing the video with a higher cognitive load using the proposed subtitle method. In the second experiment, we investigated subtitles considering connected speech form in English with 18 non-native English speakers. The results revealed that the proposed method, considering connected speech form, demonstrated an improvement in cognitive load during video viewing but it remained higher than that of regular subtitles.

Abstract Typical ductile materials are metals, which deform by the motion of defects like dislocations in association with non-directional metallic bonds. Unfortunately, this textbook mechanism does not operate in most inorganic semiconductors at ambient temperature, thus severely limiting the development of much-needed flexible electronic devices. We found a shear-deformation mechanism in a recently discovered ductile semiconductor, monoclinic-silver sulfide (Ag2S), which is defect-free, omni-directional, and preserving perfect crystallinity. Our first-principles molecular dynamics simulations elucidate the ductile deformation mechanism in monoclinic-Ag2S under six types of shear systems. Planer mass movement of sulfur atoms plays an important role for the remarkable structural recovery of sulfur-sublattice. This in turn arises from a distinctively high symmetry of the anion-sublattice in Ag2S, which is not seen in other brittle silver chalcogenides. Such mechanistic and lattice-symmetric understanding provides a guideline for designing even higher-performance ductile inorganic semiconductors.

Recent discoveries of polar topological structures (e.g., skyrmions and merons) in ferroelectric/paraelectric heterostructures have opened a new field of polar topotronics. However, how complex interplay of photoexcitation, electric field and mechanical strain controls these topological structures remains elusive. To address this challenge, we have developed a computational approach at the nexus of machine learning and first-principles simulations. Our multiscale neural-network quantum molecular dynamics molecular mechanics approach achieves orders-of-magnitude faster computation, while maintaining quantum-mechanical accuracy for atoms within the region of interest. This approach has enabled us to investigate the dynamics of vortex states formed in PbTiO3 nanowires embedded in SrTiO3. We find topological switching of these vortex states to topologically trivial, uniformly polarized states using electric field and trivial domain-wall states using shear strain. These results, along with our earlier results on optical control of polar topology, suggest an exciting new avenue toward opto-electro-mechanical control of ultrafast, ultralow-power polar topotronic devices.

Abstract We report on the design and evaluation of a flashcard application, enhanced with emotional binaural narration to support second language (L2) vocabulary learning. Typically, the voice narration used in English vocabulary learning is recorded by native speakers with a standard accent to ensure accurate pronunciation and clarity. However, a clear but monotonous narration may not always aid learners in retaining new vocabulary items in their semantic memory. As such, enhancing textual flashcards with emotional narration in the learner’s native language can foster the retention of new L2 words in episodic rather than semantic memory as greater emotive expression reinforces episodic memory retention. We evaluated the effects of binaural emotive narration with traditional textual flashcards on L2 word retention (immediate and delayed) in laboratory experiments with native Japanese-speaking English learners. Our results suggest that the learners were able to retain approximately 60% more L2 words long-term with the proposed approach compared to traditional flashcards.

Owing to the pervasive use of displays and smartphones, mobile interactions with display screens have gained attention within the advertising and gaming industries as well as in human-computer interaction research. Communication through QR code-like markers and localization via AR markers are common examples of such interactions. However, these visible markers interfere with the display content; this problem is critical for localization over a wide range of interactions, and fewer markers result in less reliability and accuracy. Although some studies have addressed this issue, few have focused on near-screen interaction without additional hardware. To address this problem, we propose an easy-to-install localization method that uses an array of AR markers, which are made imperceptible to the human eye through chromaticity vibration at 30 Hz. We mainly focus on applications, such as digital signage, where users point their smartphones at the display content. Through four evaluations, we confirm that the pointing error is within 1 mm, and that the proposed system works, when the distance between the screen and smartphone is 4-24 times the size of the AR marker. In addition, we establish that our system is robust against rotation. Finally, we present two potential application scenarios, advertising and navigation.

Invisible-light communication between a projector and multiple devices on the projection plane enables human-computer interaction applications such as movable displays and pointing devices. In this paper, we propose a pixel-level infrared light communication (PILC) projector that can project rich, invisible position-dependent information on high-contrast, full-color visible images. Although previous methods for such communication sacrificed the image quality, the amount of information, or other features, our method meets all the requirements by adding an infrared light source to a full-color DLP projector. This system alternately projects visible images and invisible data sequences with visible and infrared light, respectively. As proof of concept, we built a functional prototype and confirmed that its image contrast is more than 60 times that of a previous method while maintaining pixel-level position accuracy. In addition, we investigated flicker-perception rates in response to different amounts of embedded information and discovered that we could send rich information (up to approximately 2 kbps) with imperceptible flicker. Finally, we built a tabletop application with the proposed system, in which users can simultaneously see both an aerial photograph and a map through movable displays.

Collaboration between computer graphics and multiple robots has attracted increasing attention in several fields. To enhance the seamless connection between them, the system should be able to accurately determine the position and state of the robots and to control them easily and instantly. However, realizing a responsive control system for a large number of mobile robots without complicated settings while avoiding the system load problem is not trivial. We propose a novel system, called “Phygital Field,” for the localization and control of multiple mobile robots. Utilizing pixel-level visible light communication technology, our system can project two types of information in the same location: visible images for humans and data patterns for mobile robots. The system uses coded light superimposed onto a visual image and projected onto the robots. The robots localize their position by receiving and decoding the projected light and can follow a target using the coded velocity vector field. Localization and control information can be independently conveyed in each pixel, and we can change this information over time. The system only requires a projector to control the robot swarm; thus, it can be used on any projection surface. We experimentally assess the localization accuracy of our system for both stationary and moving robots. To further illustrate the utility of our proposed system, we demonstrate the control of multiple mobile robots in spatially and temporally varying vector fields. We also propose prototype applications that can provide users with novel content from collaboration between computer graphics and robot swarm.

QXMD is a scalable, parallel program for Quantum Molecular Dynamics simulations with various eXtensions. Its simulation engine is based on (time-dependent) density functional theory using pseudopotentials and a plane-wave basis set, while extensions include nonadiabatic electron–nuclei dynamics and multiscale shock technique. QXMD serves as a community-development platform for new methods and algorithms, a research platform on high-end parallel supercomputers, and an educational platform for hands-on training. Keywords: Nonadiabatic quantum molecular dynamics, Parallel computing, Hands-on training

Dielectric polymers such as polyethylene (PE) have a wide range of energy and electronic applications. While recent studies have shown significant effects of chemical defects on the electronic structure of PE, those on the dielectric properties remain elusive. Here, first-principles quantum-mechanical calculations show anisotropic dielectric constants of PE, which are sensitive to the type of defects. Specifically, addition of iodine defects increases the high-frequency dielectric constant. Addition of hydroxyl or carboxyl group, on the other hand, causes noticeable anisotropic changes in the static dielectric constant, which is well elucidated through the rotation and concerted motions of chemical groups. The sensitivity of these defects may be exploited to rationally alter the behavior of PE.

The article offers a Retrieval-Augmented-Generation (RAG)-enhanced dense video captioning system aimed at improving human activity recognition indoors. Topics include the limitations of current video captioning methods in disambiguating similar actions in restricted field-of-view settings, the integration of 3D reconstruction and detection techniques to build an environmental knowledge base, and the use of a large language model to enhance captions by incorporating spatial information.

The article offers a framework called WatchLogger for keystroke detection and recognition of typed words using a smartwatch's sensors, addressing potential privacy risks associated with wearable devices. Topics include the use of audio and accelerometer signals for detecting typed words, the development of an ensemble classification model to improve word recognition accuracy, and the creation of the WTW-100 dataset for evaluating performance.