Search

The extensive research on wearable devices has led to devices with various shapes and mounting locations. Wearable devices are often used to record the user’s biometric information, and methods have been proposed to detect physical abnormalities from the acquired data. Among various kinds of biometric data, pulse data has been used in methods such as heart rate monitoring and emotion recognition. The most common type of pulse sensor uses photoplethysmography (PPG), which irradiates a green LED on the skin and measures pulse data from changes in the light reflected from blood vessels. PPG sensors have been implemented in commercially available wearable devices such as smartwatches. However, a PPG sensor requires blood flow for data acquisition, and when a smartwatch is worn on an artificial body part such as a prosthetic hand or a robotic arm, data cannot be acquired because there is no blood flow. In this study, we propose a method that enables a PPG sensor to measure arbitrary pulse data by using a display. If this method is successful, it will enable pulse data measured at the junction of a living limb and an artificial limb to be input to the display; then, a smartwatch attached to the artificial limb will read the same pulse data. In this paper, we focus on the heart rate and report the results of an experiment in which a target heart rate was input and the display was controlled accordingly to determine whether the target heart rate could be obtained by a smartwatch. We implemented a display drawing program and conducted the evaluation with five kinds of smartwatches and four kinds of displays. The results showed that the error between the target heart rate and the heart rate acquired by the smartwatch was within 3 beats per minute (bpm) in many cases when the target heart rate was set to 60–100 bpm. When the target heart rate was set to 40–55 and 105–200 bpm, the heart rate could also be input into the smartwatch with a small error under certain conditions. Moreover, when generated PPG data was imported into heart rate variability (HRV) analysis software, it was recognized as a pulse wave in the same way as real PPG data obtained from a person. We compared the heart rate, RR interval, and SDNN calculated from the real and generated PPG data, and we confirmed that the proposed method could generate stable PPG data. On the other hand, when the waveforms were compared, the generated PPG waveform differed greatly from the real PPG waveform, which indicated that the software could calculate the heart rate, RR interval, SDNN, and LF/HF ratio regardless of the waveform. This result suggests that the calculation of these parameters without verifying the waveform would be vulnerable to an attack with fake PPG data.

Sensor data has been used in social security and welfare infrastructures such as insurance and medical care to provide personalized products and services; there is a risk that attackers can alter sensor data to obtain unfair benefits. We consider that one of the attack methods to modify sensor data is to attack the wearer’s body to modify biometric information. In this study, we propose a noninvasive attack method to modify the sensor value of a photoplethysmogram. The proposed method can disappear pulse wave peaks by pressurizing the upper arm with air pressure to control blood volume. Seven subjects experiencing a rest environment and five subjects experiencing an after-exercise environment wore five different models of smartwatches, and three pressure patterns were performed. It was confirmed in both situations that the displayed heart rate decreased from the true heart rate.

Delays (wait times) often occur in computer use, and an illusion changing the subjective elapsed time can occur unintentionally due to perceptual stimuli from computer interfaces. Since this can adversely affect users’ experience and state of mind, investigating and manipulating such an illusion with computer interfaces has been an important topic for many years. While many studies have focused on visual interfaces, few studies examine these two research directions: 1) the possibility that illusion-manipulation methods verified under specific conditions may cause illusions to have completely different trends due to certain factors and 2) the feasibility of illusion-manipulation methods for changing the subjective elapsed time by using only visual stimuli in a small space at the edge of the field of view/screen. Therefore, this study verified these two directions. We designed two methods for changing the frequency and duration of visual stimuli. We also designed two visualization styles: the blinking visualization style and vibration visualization style. Three evaluations were conducted. First, the results showed the feasibility of illusion-manipulation methods using visual stimuli in a small space at the edge of the field of view/screen. The subjective elapsed time changed with a constant trend from −8.9 to 0.6% in 10 seconds. Second, the results showed that the trend in elapsed time of illusions caused by the same visual stimulus pattern could change when different visualization styles were used. Specifically, when using the method of changing the duration of visual stimuli, the increase or decrease in the subjective elapsed time was reversed depending on whether the blinking or visualization style was used. Third, the results showed that the trend in elapsed time of illusions caused by visual stimuli could change when there is visual information other than illusion-inducing visual stimuli (i.e., when some of the user’s attention is diverted to something other than illusion-inducing visual stimuli). Specifically, when using the method of changing the frequency of visual stimuli, the increase or decrease in subjective elapsed time was reversed depending on whether there was visual information other than illusion-inducing visual stimuli (e.g., an advertisement video, a first-person perspective movie of walking outdoors). These results could provide new insights for designers, researchers, and users to explore and consider the illusion of changing the subjective elapsed time with visual interfaces.

Systems of presenting myoelectricity sensor information allow users to understand the body's load for various purposes, such as medical rehabilitation and sports training. If there is a method to create the psychological phenomenon of unconsciously increasing or decreasing a user's load perception simply by changing how to present the myoelectricity sensor values, it will help design a more effective system. Therefore, we propose a method to manipulate load perception by presenting modified myoelectricity sensor information. The proposed method aims to induce higher or lower load perception by modifying the actual myoelectric value to a higher or lower value. We implemented a prototype system and evaluated our method for the two types of load perception of weight perception and fatigue perception when handling objects. The result showed that most subjects unconsciously increased or decreased their load perception to match the presented myoelectric value, while the minority subjects got the opposite response from the majority subjects. This result indicates the feasibility of user assistance systems that use this phenomenon for a good purpose, such as systems that slightly reduce the load perception during physical activity. On the other hand, this result also indicates the feasibility of systems that use this phenomenon for a bad purpose, such as systems that increase user's fatigue to harm user's activity. This study provides helpful findings for designing and using sensor information presentation systems considering the psychological phenomenon.

The electromyogram (EMG) is a waveform representation of the action potential generated by muscle cells using electrodes. EMG acquired using surface electrodes is called surface EMG (sEMG), and it is the acquisition of muscle action potentials transmitted by volume conduction from the skin. Surface electrodes require disposable conductive gel or adhesive tape to be attached to the skin, which is costly to run, and the tape is hard on the skin when it is removed. Muscle activity can be evaluated by acquiring muscle potentials and analyzing quantitative, temporal, and frequency factors. It is also possible to evaluate muscle fatigue because the frequency of the EMG becomes lower as the muscle becomes fatigued. Research on human activity recognition from EMG signals has been actively conducted and applied to systems that support arm and hand functions. This paper proposes a method for recognizing the muscle activity state of the arm using pulse wave data (PPG: Photoplethysmography) and a method for estimating EMG using pulse wave data. This paper assumes that the PPG sensor is worn on the user’s wrist to measure the heart rate. The user also attaches an elastic band to the upper arm, and when the user exerts a force on the arm, the muscles of the upper arm contract. The arteries are then constricted, and the pulse wave measured at the wrist becomes weak. From the change in the pulse wave, the muscle activity of the arm can be recognized and the number of action potentials of the muscle can be estimated. From the evaluation experiment with five subjects, three types of muscle activity were recognized with 80+%, and EMG was estimated with approximately 20% error rate.

Artificial Intelligence (AI) for human emotion estimation, such as facial emotion estimation, has been actively studied. On the other hand, there has been little research on unconscious phenomena in cognition and psychology (i.e., cognitive biases) caused by viewing AI emotion estimation information. Therefore, this study verifies RQ “Do people have a cognitive bias in which impressions of others (i.e., how to see and feel about others) are changed by viewing biased AI’s emotion estimation information? If it exists, can impression manipulation methods that intentionally use this cognitive bias be realized?” The proposed method for verification makes the emotion estimation system biased so as to estimate emotion more positively/negatively than AI without bias. A prototype system was implemented. Evaluation using video showed that the presentation of biased emotion estimation information causes a phenomenon that quickly and unconsciously changes the way people see and feel others’ impressions, which supported the RQ. Specifically, viewing information that estimated others’ emotions more positively/negatively caused the phenomenon in which the user’s self-judgment was overridden and others’ impressions of emotions, words, and actions were perceived more positively/negatively. The existence of this phenomenon and method indicates that biased emotion estimation AI has the potential to both cause adverse effects on people and support people for good purposes through the manipulation of their impressions. This study provides helpful insights for the design and use of emotion estimation AI considering cognitive biases.

The Sussex-Huawei Locomotion-Transportation (SHL) Recognition Challenges aim to advance and capture the state-of-the-art in locomotion and transportation mode recognition from smartphone motion (inertial) sensors. The goal of this series of machine learning and data science challenges was to recognize eight locomotion and transportation activities (Still, Walk, Run, Bus, Car, Train, Subway). The three challenges focused on time-independent (SHL 2018), position-independent (SHL 2019) and user-independent (SHL 2020) evaluations, respectively. Overall, we received 48 submissions (out of 93 teams who registered interest) involving 201 scientists over the three years. The survey captures the state-of-the-art through a meta-analysis of the contributions to the three challenges, including approaches, recognition performance, computational requirements, software tools and frameworks used. It was shown that state-of-the-art methods can distinguish with relative ease most modes of transportation, although the differentiating between subtly distinct activities, such as rail transport (Train and Subway) and road transport (Bus and Car) still remains challenging. We summarize insightful methods from participants that could be employed to address practical challenges of transportation mode recognition, for instance, to tackle over-fitting, to employ robust representations, to exploit data augmentation, and to exploit smart post-processing techniques to improve performance. Finally, we present baseline results to compare the three challenges with a unified recognition pipeline and decision window length.

There are situations where manipulating subjective time would be desirable, such as reducing waiting time, and there are many studies to manipulate subjective time. However, it is not easy to use previous methods in various situations because most of them use visual and auditory information. This study proposes a method to manipulate the subjective time by the tactile stimuli from wrist-worn devices. We designed three types of tactile stimuli presentation methods that change the number, the duration, and the time interval of the stimuli. The evaluation result clarified the elements of the tactile stimuli that intentionally changed the subjective time and confirmed that our method can change the subjective time by about 23% (from −6% to +17%). Since few studies have focused on the phenomenon in which the subjective time changes depending on the tactile stimuli from information devices, our findings can contribute to designing information devices and user experiences.

With the increasing use of wearable devices equipped with various sensors, information on human activities, biometrics, and surrounding environments can be obtained via sensor data at any time and place. When such devices are attached to arbitrary body parts and multiple devices are used to capture body-wide movements, it is important to estimate where the devices are attached. In this study, we propose a method that estimates the load positions of wearable devices without requiring the user to perform specific actions. The proposed method estimates the time difference between a heartbeat obtained by an ECG sensor and a pulse wave obtained by a pulse sensor, and it classifies the pulse sensor position from the estimated time difference. Data were collected at 12 body parts from four male subjects and one female subject, and the proposed method was evaluated in both user-dependent and user-independent environments. The average F-value was 1.0 when the number of target body parts was from two to five.