Search

The violin is one of the most popular musical instruments, but mastering it requires a significant amount of practice time. The bowing action (pressure, position, speed) of the right hand is crucial in determining tonal quality, but this is difficult to master. This study compared the bowing movements, specifically bow pressure, bow position, and bow speed, of experienced players with those of beginners. Identifying common bowing characteristics of experienced violin players can aid the evaluation of beginners' skills and the development of a feedback system that supports self-practice and instruction., Comment: Part of proceedings of 6th International Conference AsiaHaptics 2024

Wearable robots are designed to be worn on the human body. Taking advantage of their physical form, various applications for wearable robots are being considered. This study proposes a wearable robot worn on the abdomen and a new interaction with it. Our robot enables a variety of applications related to communication between the wearer and surrounding humans through visual and tactile feedback. The contributions of this research will be (1) the proposal of a novel wearable robot worn on the abdomen and (2) a new interaction with it., Comment: Part of proceedings of 6th International Conference AsiaHaptics 2024

A precoded orthogonal time frequency space (OTFS) modulation scheme relying on faster-than-Nyquist (FTN) transmission over doubly selective fading channels is {proposed}, which enhances the spectral efficiency and improves the Doppler resilience. We derive the input-output relationship of the FTN signaling in the delay-Doppler domain. Eigenvalue decomposition (EVD) is used for eliminating both the effects of inter-symbol interference and correlated additive noise encountered in the delay-Doppler domain to enable efficient symbol-by-symbol demodulation. Furthermore, the power allocation coefficients of individual frames are optimized for maximizing the mutual information under the constraint of the derived total transmit power. Our performance results demonstrate that the proposed FTN-based OTFS scheme can enhance the information rate while achieving a comparable BER performance to that of its conventional Nyquist-based OTFS counterpart that employs the same root-raised-cosine shaping filter., Comment: 5 pages, 3 figures

We present a realistic lattice QCD study on low-energy $N$-$J/\psi$ and $N$-$\eta_c$ interactions based on (2+1) flavor configurations with nearly physical pion mass $m_\pi=146$ MeV. The interactions, extracted from the spacetime correlations of nucleon and charmonium system by using the HAL QCD method, are found to be attractive in all distances and possess a characteristic long-range tail consistent with the two-pion exchange potential. The resulting $S$-wave scattering lengths are $0.30(2)\left(^{+0}_{-2}\right)$ fm, $0.38(4)\left(^{+0}_{-3}\right)$ fm, and $0.21(2)\left(^{+0}_{-1}\right)$ fm for spin-$3/2$ $N$-$J/\psi$, spin-$1/2$ $N$-$J/\psi$, and spin-$1/2$ $N$-$\eta_c$, respectively. Our results are orders of magnitude larger than those from the photoproduction experiments assuming the vector meson dominance. Our findings may provide deeper understanding of the nonperturbative QCD phenomena ranging from the origin of nucleon mass to the in-medium $J/\psi$ mass modification as well as the properties of hidden-charm pentaquark states., Comment: v2, minor modifications are made to match with the published version

To develop high-performing Visual Language Models (VLMs), it is essential to prepare multimodal resources, such as image-text pairs, interleaved data, and instruction data. While multimodal resources for English are abundant, there is a significant lack of corresponding resources for non-English languages, such as Japanese. To address this problem, we take Japanese as a non-English language and propose a method for rapidly creating Japanese multimodal datasets from scratch. We collect Japanese image-text pairs and interleaved data from web archives and generate Japanese instruction data directly from images using an existing VLM. Our experimental results show that a VLM trained on these native datasets outperforms those relying on machine-translated content., Comment: 15 pages, 7 figures

Tactile sensation is essential for humans to recognize objects. Various devices have been developed in the past for tactile presentation by electrostatic force, which are easy to configure devices, but there is currently no such device that features stretchability. Considering that the device is worn over the joints of a human body or robot, it is extremely important that the device itself be stretchable. In this study, we propose a stretchable electrostatic tactile surface comprising a stretchable transparent electrode and a stretchable insulating film that can be stretched to a maximum of 50%. This means that when attached to the human body, this surface can respond to the expansion and contraction that occur due to joint movements. This surface can also provide tactile information in response to deformation such as pushing and pulling. As a basic investigation, we measured the lower limit of voltage that can be perceived by changing the configuration of the surface and evaluated the states of stretching and contraction. We also investigated and modeled the relationship between the voltage and the perceived intensity., Comment: 7 pages, 9 figures

Accurately instructing posture and the position of the body's center of gravity is challenging. In this study, we propose a system that utilizes haptic feedback to induce the Center of Pressure (CoP) movement. The Wii Balance Board is employed to sense the CoP, and vibration motors are used for haptic feedback. To provide a comparison, inductions were also performed using visual and auditory feedback, and the time required for induction was measured. Additionally, after the experiments, a questionnaire survey was conducted., Comment: 4 pages, 6 figures

In this research, we aim to realize cushion interface for operating smart home. We designed user-defined gestures using cushion and developed gesture recognition system. We asked some users to make gestures using cushions for operating home appliances and determined user-defined gesture sets. We developed two methods for gesture identification. The First, We inserted sensor modules consisting of photo reflective sensors and acceleration sensor inside a cushion. The second, we embedded the acceleration sensor arrays in the cushion cover. Gesture recognizer was implemented using Convolutional Neural Networks (CNN). To evaluate our method, We conducted an experiment to measure recognition accuracy. Results showed that an average accuracy was 94.8% when training for each user, and an average accuracy of 91.3% when testing with a user that did not exist in the training data set., Comment: 10 pages

In this study, we consider the problem of predicting task success for open-vocabulary manipulation by a manipulator, based on instruction sentences and egocentric images before and after manipulation. Conventional approaches, including multimodal large language models (MLLMs), often fail to appropriately understand detailed characteristics of objects and/or subtle changes in the position of objects. We propose Contrastive $\lambda$-Repformer, which predicts task success for table-top manipulation tasks by aligning images with instruction sentences. Our method integrates the following three key types of features into a multi-level aligned representation: features that preserve local image information; features aligned with natural language; and features structured through natural language. This allows the model to focus on important changes by looking at the differences in the representation between two images. We evaluate Contrastive $\lambda$-Repformer on a dataset based on a large-scale standard dataset, the RT-1 dataset, and on a physical robot platform. The results show that our approach outperformed existing approaches including MLLMs. Our best model achieved an improvement of 8.66 points in accuracy compared to the representative MLLM-based model., Comment: Accepted for presentation at CoRL2024

We propose a quantum hash function based on Gaussian boson sampling on a photonic quantum computer, aiming to provide quantum-resistant security. Extensive simulations demonstrate that this hash function exhibits strong properties of preimage, second preimage, and collision resistance, which are essential for cryptographic applications. Notably, the estimated number of attempts required for a successful collision attack increases exponentially with the mode counts of the photonic quantum computer, suggesting robust resistance against birthday attacks. We also analyze the sampling cost for physical implementation and discuss potential applications to blockchain technologies, where the inherent quantum nature of the hash computation could provide quantum-resistant security. The high dimensionality of the quantum state space involved in the hashing process poses significant challenges for quantum attacks, indicating a path towards quantum security. Our work lays the foundation for a new paradigm of quantum-resistant hashing with applications in emerging quantum-era information systems., Comment: 9+3 pages, 5+2 figures

In this work, we address the challenge of developing automatic evaluation metrics for image captioning, with a particular focus on robustness against hallucinations. Existing metrics are often inadequate for handling hallucinations, primarily due to their limited ability to compare candidate captions with multifaceted reference captions. To address this shortcoming, we propose DENEB, a novel supervised automatic evaluation metric specifically robust against hallucinations. DENEB incorporates the Sim-Vec Transformer, a mechanism that processes multiple references simultaneously, thereby efficiently capturing the similarity between an image, a candidate caption, and reference captions. To train DENEB, we construct the diverse and balanced Nebula dataset comprising 32,978 images, paired with human judgments provided by 805 annotators. We demonstrated that DENEB achieves state-of-the-art performance among existing LLM-free metrics on the FOIL, Composite, Flickr8K-Expert, Flickr8K-CF, Nebula, and PASCAL-50S datasets, validating its effectiveness and robustness against hallucinations., Comment: ACCV 2024

In recent years, virtual sensing techniques have been extensively studied as a method of data collection in simulated virtual spaces for the development of human activity recognition (HAR) systems. To date, this technique has enabled the transformation between different modalities, significantly expanding datasets that are typically difficult to collect. However, there is limited research on how to make virtual sensors more easy-to-use or effective as tools for making sense of the sensor data. The context-awareness and intuitiveness of XR make it an ideal platform for virtual sensors. In this work, we demonstrate, Vsens Reality, the use of virtual sensors under the XR context as an augmentation tool for the design of interactive systems.

In this study, we aim to develop a domestic service robot (DSR) that, guided by open-vocabulary instructions, can carry everyday objects to the specified pieces of furniture. Few existing methods handle mobile manipulation tasks with open-vocabulary instructions in the image retrieval setting, and most do not identify both the target objects and the receptacles. We propose the Dual-Mode Multimodal Ranking model (DM2RM), which enables images of both the target objects and receptacles to be retrieved using a single model based on multimodal foundation models. We introduce a switching mechanism that leverages a mode token and phrase identification via a large language model to switch the embedding space based on the prediction target. To evaluate the DM2RM, we construct a novel dataset including real-world images collected from hundreds of building-scale environments and crowd-sourced instructions with referring expressions. The evaluation results show that the proposed DM2RM outperforms previous approaches in terms of standard metrics in image retrieval settings. Furthermore, we demonstrate the application of the DM2RM on a standardized real-world DSR platform including fetch-and-carry actions, where it achieves a task success rate of 82% despite the zero-shot transfer setting. Demonstration videos, code, and more materials are available at https://kkrr10.github.io/dm2rm/.

Large optical reconfigurable intelligent surfaces (ORISs) are proposed for employment on building rooftops to facilitate free-space quantum key distribution (QKD) between high-altitude platforms (HAPs) and low-altitude platforms (LAPs). Due to practical constraints, the communication terminals can only be positioned beneath the LAPs, preventing direct upward links to HAPs. By deploying ORISs on rooftops to reflect the beam arriving from HAPs towards LAPs from below, reliable HAP-to-LAP links can be established. To accurately characterize the optical beam propagation, we develop an analytical channel model based on extended Huygens-Fresnel principles for representing both the atmospheric turbulence effects and the hovering fluctuations of LAPs. This model facilitates adaptive ORIS beam-width control through linear, quadratic, and focusing phase shifts, which are capable of effectively mitigating the detrimental effects of beam broadening and pointing errors (PE). Furthermore, we derive a closed-form expression for the information-theoretic bound of the QKD secret key rate (SKR) of the HAP-to-LAP links. Our findings demonstrate that quadratic phase shifts enhance the SKR at high HAP-ORIS zenith angles or mild PE conditions by narrowing the beam to optimal sizes. By contrast, linear phase shifts are advantageous at low HAP-ORIS zenith angles under moderate-to-high PE by diverging the beam to mitigate LAP fluctuations., Comment: 16 pages, 7 figures, 3 tables

This paper provides an efficient solution approach to the screen-line counter location problem (SCLP), which is a counter location problem with the constraint that the traffic between OD pairs must be observed at least once. This paper formulates the SCLP using a graph cut approach, which consists of an enumeration of cuts and a cut selection problem. These problems can be reduced to a concise formulation that extends the maximum weight closure problem for two problems: finding the minimum number of links that observe all OD pairs and finding the maximum number of observed OD pairs with a budget-constrained number of links. Insights into the characteristics of cuts give superior upper bounds on the problem of finding the minimum number of links that observe all OD pairs. The proposed method is evaluated on the Sioux-Falls network. It shows that it is possible to derive a solution equivalent to the optimal solution found in previous studies in a very short computation time., Comment: 16 pages, 9 figures

Domestic service robots (DSRs) that support people in everyday environments have been widely investigated. However, their ability to predict and describe future risks resulting from their own actions remains insufficient. In this study, we focus on the linguistic explainability of DSRs. Most existing methods do not explicitly model the region of possible collisions; thus, they do not properly generate descriptions of these regions. In this paper, we propose the Nearest Neighbor Future Captioning Model that introduces the Nearest Neighbor Language Model for future captioning of possible collisions, which enhances the model output with a nearest neighbors retrieval mechanism. Furthermore, we introduce the Collision Attention Module that attends regions of possible collisions, which enables our model to generate descriptions that adequately reflect the objects associated with possible collisions. To validate our method, we constructed a new dataset containing samples of collisions that can occur when a DSR places an object in a simulation environment. The experimental results demonstrated that our method outperformed baseline methods, based on the standard metrics. In particular, on CIDEr-D, the baseline method obtained 25.09 points, whereas our method obtained 33.08 points., Comment: Accepted for presentation at Advanced Robotics 24

The transparent formulation of explanation methods is essential for elucidating the predictions of neural networks, which are typically black-box models. Layer-wise Relevance Propagation (LRP) is a well-established method that transparently traces the flow of a model's prediction backward through its architecture by backpropagating relevance scores. However, the conventional LRP does not fully consider the existence of skip connections, and thus its application to the widely used ResNet architecture has not been thoroughly explored. In this study, we extend LRP to ResNet models by introducing Relevance Splitting at points where the output from a skip connection converges with that from a residual block. Our formulation guarantees the conservation property throughout the process, thereby preserving the integrity of the generated explanations. To evaluate the effectiveness of our approach, we conduct experiments on ImageNet and the Caltech-UCSD Birds-200-2011 dataset. Our method achieves superior performance to that of baseline methods on standard evaluation metrics such as the Insertion-Deletion score while maintaining its conservation property. We will release our code for further research at https://5ei74r0.github.io/lrp-for-resnet.page/, Comment: Accepted for presentation at ECCV2024

Rearranging objects (e.g. vase, door) back in their original positions is one of the most fundamental skills for domestic service robots (DSRs). In rearrangement tasks, it is crucial to detect the objects that need to be rearranged according to the goal and current states. In this study, we focus on Rearrangement Target Detection (RTD), where the model generates a change mask for objects that should be rearranged. Although many studies have been conducted in the field of Scene Change Detection (SCD), most SCD methods often fail to segment objects with complex shapes and fail to detect the change in the angle of objects that can be opened or closed. In this study, we propose a Co-Scale Cross-Attentional Transformer for RTD. We introduce the Serial Encoder which consists of a sequence of serial blocks and the Cross-Attentional Encoder which models the relationship between the goal and current states. We built a new dataset consisting of RGB images and change masks regarding the goal and current states. We validated our method on the dataset and the results demonstrated that our method outperformed baseline methods on $F_1$-score and mean IoU., Comment: Accepted for publication in Advanced Robotics

This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its activities, and technical reports on the LLMs developed by LLM-jp. For the latest activities, visit https://llm-jp.nii.ac.jp/en/.

We consider the task of generating segmentation masks for the target object from an object manipulation instruction, which allows users to give open vocabulary instructions to domestic service robots. Conventional segmentation generation approaches often fail to account for objects outside the camera's field of view and cases in which the order of vertices differs but still represents the same polygon, which leads to erroneous mask generation. In this study, we propose a novel method that generates segmentation masks from open vocabulary instructions. We implement a novel loss function using optimal transport to prevent significant loss where the order of vertices differs but still represents the same polygon. To evaluate our approach, we constructed a new dataset based on the REVERIE dataset and Matterport3D dataset. The results demonstrated the effectiveness of the proposed method compared with existing mask generation methods. Remarkably, our best model achieved a +16.32% improvement on the dataset compared with a representative polygon-based method., Comment: Accepted for presentation at IROS2024

We report scale setting and hadronic properties for our new lattice QCD gauge configuration set (HAL-conf-2023). We employ $(2+1)$-flavor nonperturbatively improved Wilson fermions with stout smearing and the Iwasaki gauge action on a $96^4$ lattice, and generate configurations of 8,000 trajectories at the physical point. We show the basic properties of the configurations such as the plaquette value, topological charge distribution and their auto-correlation times. The scale setting is performed by detailed analyses of the $\Omega$ baryon mass. We calculate the physical results of quark masses, decay constants of pseudoscalar mesons and single hadron spectra in light quark sector. The masses of the stable hadrons are found to agree with the experimental values within a sub-percent level., Comment: 32 pages and 18 figures; v2: reference(s) added; v3: accepted for publication in Phys. Rev. D; v4: typo(s) corrected, published version

In the last decade, academic and industrial researchers have focused on persistent memory because of the development of the first practical product, Intel Optane. One of the main challenges of persistent memory programming is to guarantee consistent durability over separate memory addresses, and Wang et al. proposed a persistent multi-word compare-and-swap (PMwCAS) algorithm to solve this problem. However, their algorithm contains redundant compare-and-swap (CAS) and cache flush instructions and does not achieve sufficient performance on many-core CPUs. This paper proposes a new algorithm to improve performance on many-core CPUs by removing useless CAS/flush instructions from PMwCAS operations. We also exclude dirty flags, which help ensure consistent durability in the original algorithm, from our algorithm using PMwCAS descriptors as write-ahead logs. Experimental results show that the proposed method is up to ten times faster than the original algorithm and suggests several productive uses of PMwCAS operations., Comment: 8 pages, 14 figures

Point cloud registration serves as a basis for vision and robotic applications including 3D reconstruction and mapping. Despite significant improvements on the quality of results, recent deep learning approaches are computationally expensive and power-hungry, making them difficult to deploy on resource-constrained edge devices. To tackle this problem, in this paper, we propose a fast, accurate, and robust registration for low-cost embedded FPGAs. Based on a parallel and pipelined PointNet feature extractor, we develop custom accelerator cores namely PointLKCore and ReAgentCore, for two different learning-based methods. They are both correspondence-free and computationally efficient as they avoid the costly feature matching step involving nearest-neighbor search. The proposed cores are implemented on the Xilinx ZCU104 board and evaluated using both synthetic and real-world datasets, showing the substantial improvements in the trade-offs between runtime and registration quality. They run 44.08-45.75x faster than ARM Cortex-A53 CPU and offer 1.98-11.13x speedups over Intel Xeon CPU and Nvidia Jetson boards, while consuming less than 1W and achieving 163.11-213.58x energy-efficiency compared to Nvidia GeForce GPU. The proposed cores are more robust to noise and large initial misalignments than the classical methods and quickly find reasonable solutions in less than 15ms, demonstrating the real-time performance., Comment: 27 pages, 19 figures

IoT sensors are crucial for visualizing multidimensional and multimodal information and enabling future IT applications/services such as cyber-physical space, digital twins, autonomous driving, smart cities, and virtual/augmented reality (VR or AR). However, IoT sensors need to be battery-free to realistically manage and maintain the growing number of available sensing devices. Here, we provide a novel sensor design approach that employs metasurfaces to enable multifunctional sensing without requiring an external power source. Importantly, unlike existing metasurface-based sensors, our metasurfaces can sense multiple physical parameters even at a fixed frequency by breaking classic harmonic oscillations in the time domain, making the proposed sensors viable for usage with limited frequency resources. Moreover, we provide a method for predicting physical parameters using the machine learning-based approach of random forest regression. The sensing performance was confirmed by estimating temperature and light intensity, and excellent determination coefficients larger than 0.96 were achieved. Our study affords new opportunities for sensing multiple physical properties without relying on an external power source or needing multiple frequencies, which markedly simplifies and facilitates the design of next-generation wireless communication systems., Comment: 47 pages, 23 figures

Establishing an automatic evaluation metric that closely aligns with human judgments is essential for effectively developing image captioning models. Recent data-driven metrics have demonstrated a stronger correlation with human judgments than classic metrics such as CIDEr; however they lack sufficient capabilities to handle hallucinations and generalize across diverse images and texts partially because they compute scalar similarities merely using embeddings learned from tasks unrelated to image captioning evaluation. In this study, we propose Polos, a supervised automatic evaluation metric for image captioning models. Polos computes scores from multimodal inputs, using a parallel feature extraction mechanism that leverages embeddings trained through large-scale contrastive learning. To train Polos, we introduce Multimodal Metric Learning from Human Feedback (M$^2$LHF), a framework for developing metrics based on human feedback. We constructed the Polaris dataset, which comprises 131K human judgments from 550 evaluators, which is approximately ten times larger than standard datasets. Our approach achieved state-of-the-art performance on Composite, Flickr8K-Expert, Flickr8K-CF, PASCAL-50S, FOIL, and the Polaris dataset, thereby demonstrating its effectiveness and robustness., Comment: CVPR 2024

Objective: Domestic abuse cases have risen significantly over the last four years, in part due to the COVID-19 pandemic and the challenges for victims and survivors in accessing support. In this study, we investigate the role that chatbots - Artificial Intelligence (AI) and rule-based - may play in supporting victims/survivors in situations such as these or where direct access to help is limited. Methods: Interviews were conducted with experts working in domestic abuse support services and organizations (e.g., charities, law enforcement) and the content of websites of related support-service providers was collected. Thematic content analysis was then applied to assess and extract insights from the interview data and the content on victim-support websites. We also reviewed pertinent chatbot literature to reflect on studies that may inform design principles and interaction patterns for agents used to support victims/survivors. Results: From our analysis, we outlined a set of design considerations/practices for chatbots that consider potential use cases and target groups, dialog structure, personality traits that might be useful for chatbots to possess, and finally, safety and privacy issues that should be addressed. Of particular note are situations where AI systems (e.g., ChatGPT, CoPilot, Gemini) are not recommended for use, the value of conveying emotional support, the importance of transparency, and the need for a safe and confidential space. Conclusion: It is our hope that these considerations/practices will stimulate debate among chatbots and AI developers and service providers and - for situations where chatbots are deemed appropriate for use - inspire efficient use of chatbots in the support of survivors of domestic abuse.

In this paper, we propose a method for estimating the algebraic Riccati equation (ARE) with respect to an unknown discrete-time system from the system state and input observation. The inverse optimal control (IOC) problem asks, ``What objective function is optimized by a given control system?'' The inverse linear quadratic regulator (ILQR) problem is an IOC problem that assumes a linear system and quadratic objective function. The ILQR problem can be solved by solving a linear matrix inequality that contains the ARE. However, the system model is required to obtain the ARE, and it is often unknown in fields in which the IOC problem occurs, for example, biological system analysis. Our method directly estimates the ARE from the observation data without identifying the system. This feature enables us to economize the observation data using prior information about the objective function. We provide a data condition that is sufficient for our method to estimate the ARE. We conducted a numerical experiment to demonstrate that our method can estimate the ARE with less data than system identification if the prior information is sufficient.

Transformer has been adopted to image recognition tasks and shown to outperform CNNs and RNNs while it suffers from high training cost and computational complexity. To address these issues, a hybrid approach has become a recent research trend, which replaces a part of ResNet with an MHSA (Multi-Head Self-Attention). In this paper, we propose a lightweight hybrid model which uses Neural ODE (Ordinary Differential Equation) as a backbone instead of ResNet so that we can increase the number of iterations of building blocks while reusing the same parameters, mitigating the increase in parameter size per iteration. The proposed model is deployed on a modest-sized FPGA device for edge computing. The model is further quantized by QAT (Quantization Aware Training) scheme to reduce FPGA resource utilization while suppressing the accuracy loss. The quantized model achieves 79.68% top-1 accuracy for STL10 dataset that contains 96$\times$96 pixel images. The weights of the feature extraction network are stored on-chip to minimize the memory transfer overhead, allowing faster inference. By eliminating the overhead of memory transfers, inference can be executed seamlessly, leading to accelerated inference. The proposed FPGA implementation accelerates the backbone and MHSA parts by 34.01$\times$, and achieves an overall 9.85$\times$ speedup when taking into account the software pre- and post-processing. The FPGA acceleration leads to 7.10$\times$ better energy efficiency compared to the ARM Cortex-A53 CPU. The proposed lightweight Transformer model is demonstrated on Xilinx ZCU104 board for the image recognition of 96$\times$96 pixel images in this paper and can be applied to different image sizes by modifying the pre-processing layer.