Search

To guarantee the reliability and integrity of audio, data have been focused on as an essential topic as the fast development of generative AI. Significant progress in machine learning and speech synthesis has increased the potential for audio tampering. In this paper, we focus on the digital watermarking method as a promising method to safeguard the authenticity of audio evidence. Due to the integrity of the original data with probative importance, the algorithm requires reversibility, imperceptibility, and reliability. To meet the requirements, we propose a reversible digital watermarking approach that embeds feature data concentrating in high-frequency intDCT coefficients after transforming data from the time domain into the frequency domain. We explored the appropriate hiding locations against spectrum-based attacks with novel proposed methodologies for spectral expansion for embedding. However, the drawback of fixed expansion is that the stego signal is prone to being detected by a spectral analysis. Therefore, this paper proposes two other new expansion methodologies that embed the data into variable locations—random expansion and adaptive expansion with distortion estimation for embedding—which effectively conceal the watermark’s presence while maintaining high perceptual quality with an average segSNR better than 21.363 dB and average MOS value better than 4.085. Our experimental results demonstrate the efficacy of our proposed method in both sound quality preservation and log-likelihood value, indicating the absolute discontinuity of the spectrogram after embedding is proposed to evaluate the effectiveness of the proposed reversible spectral expansion watermarking algorithm. The result of EER indicated that the adaptive hiding performed best against attacks by spectral analysis.

Conversations based on mutual intimacy are critical for maintaining positive relationships. A detailed understanding of speaker relationships in dialogues enhances various applications, such as information recommendation systems. Such systems, when interacting with multiple users, can provide more tailored information by understanding the users’ relationships. Furthermore, dialogue systems, which are becoming increasingly prevalent in society, can foster long-term user engagement by recognizing and responding to the intimacy levels of users. This study explores a method for estimating the intimacy levels of speakers and dialogue partners in conversational exchanges. Our approach utilizes a multimodal corpus of natural conversations with 71 Japanese participants, complete with metadata indicating each speaker’s perceived intimacy level. We identified key features for estimating intimacy by analyzing the statistical parameters of these features. Our comprehensive analysis encompassed both verbal and non-verbal information, including prosody, gestures, and facial expressions. The proposed intimacy estimation model combines multimodal features using a multi-stream Bi-directional Long Short-Term Memory (BLSTM) network and grasps the contextual information of conversations with a Context BLSTM. Our model’s effectiveness is demonstrated through comparisons with several baseline models. Experimental results show that our proposed model significantly improves the overall performance compared with other models. Although the RoBERTa-based method (the best baseline model) achieved an F1 score of 0.571, our method had an F1 score of 0.594. In particular, an ablation study shows that combining verbal and non-verbal features is useful for intimacy estimation. The performance was further improved by extending the dialogue context, showing that the proposed model can estimate three levels of intimacy with an F1 score of 0.666 by observing eight utterance exchanges.

Training task-oriented dialog agents based on reinforcement learning is time-consuming and requires a large number of interactions with real users. How to grasp dialog policy within limited dialog experiences remains an obstacle that makes the agent training process less efficient. In addition, most previous frameworks start training by randomly choosing training samples, which differs from the human learning method and hurts the efficiency and stability of training. Therefore, we propose Scheduled Curiosity-Deep Dyna-Q (SC-DDQ), a curiosity-driven curriculum learning framework based on a state-of- the-art model-based reinforcement learning dialog model, Deep Dyna-Q (DDQ). Furthermore, we design ed learning schedules for SC-DDQ and DDQ, respectively, following two opposite training strategies: classic curriculum learning and its reverse version. Our results show that by introducing scheduled learning and curiosity, the new framework leads to a significant improvement over the DDQ and Deep Q-learning (DQN). Surprisingly, we found that traditional curriculum learning was not always effective. Specifically, according to the experimental results, the easy-first and difficult-first strategies are more suitable for SC-DDQ and DDQ. To analyze our results, we adopt ed the entropy of sampled actions to depict action exploration and found that training strategies with high entropy in the first stage and low entropy in the last stage lead to better performance.

Task-oriented dialog policy learning is often formulated as a Reinforcement Learning problem whose rewards from the environment are extremely sparse, which means that the agent will often not find the reward by acting randomly. Thus, exploration techniques are of primary importance when solving RL problems, and more sophisticated exploration methods must be devised. In this study, we propose a replaceable curiosity-driven candidate agent exploration approach to encourage the agent to balance action sampling and explore new environments without overly violating dialog strategies. In this framework, we follow the employment of the curiosity model but design weight for the curiosity reward to balance exploration and exploitation. We designed a multi-candidate agent mechanism to filter an agent with relatively balanced action sampling for formal dialog training to motivate agents to escape pseudo-optimal actions in the early training stage. In addition, we propose a replacement mechanism for the first time to prevent the elected agents from performing poorly in the later stages of training and to fully utilize all the candidate agents. The experimental results show that the adjustable curiosity reward promotes dialog policy convergence. The agent replacement mechanism effectively blocks the training of poorly trained agents, significantly increasing the task’s average success rate and reducing the number of dialog turns. In this research, an exploration approach for task-oriented dialog system is designed to encourage agents to explore environment through balanced action sampling, without significantly deviating from learned dialog strategies. Compared to baselines, the replaceable curiosity-driven candidate agent exploration approach yields a higher average success rate of 0.714 and a lower number of average turns of 20.6.

This paper proposes a novel meta-transfer learning method to improve automatic speech recognition (ASR) performance in low-resource languages. Nowadays, we are witnessing high interest in low-resource ASR tasks aiming at delivering feasible and reliable systems with very limited data. The main challenge is the design and development of a methodology to address the issue of data scarcity. Our proposed meta-transfer learning approach combines two well-known machine-learning methods: transfer learning and meta-learning. We propose their integration that can ameliorate the training bottlenecks and overfitting issues with pre-training models on low-resource speech data. For evaluation, we conduct extensive multilingual ASR experiments on the Common Voice Corpus and Globalphone Corpus and compare the performance of the meta-transfer learning, meta-learning, and transfer learning methods. The proposed meta-transfer learning achieves a relative character error rate (CER) reduction of 11.62% over meta-learning and a relative CER reduction of 10.86% over transfer learning in low-resource experiments. We used less than 15 minutes of data for each target language in near-zero resource language experiments. Our meta-transfer learning approach achieved an average CER of 25.25% less than meta-learning and transfer learning. These results clearly demonstrate that the proposed integration works well in ASR tasks in languages with very limited data resources.

This paper proposes a novel tour guide robot, “ASAHI ReBorn”, which can lead a guest by hand one-on-one while maintaining a proper distance from the guest. The robot uses a stretchable arm interface to hold the guest’s hand and adjusts its speed according to the guest’s pace. The robot also follows a given guide path accurately using the Robot Side method, a robot navigation method that follows a pre-defined path quickly and accurately. In addition, a control method is introduced that limits the angular velocity of the robot to avoid the robot’s quick turn while guiding the guest. We evaluated the performance and usability of the proposed robot through experiments and user studies. The tour-guiding experiment revealed that the proposed method that keeps distance between the robot and the guest using the stretchable arm enables the guests to look around the exhibits compared with the condition where the robot moved at a constant velocity.

This paper aims to explore an alternative reversible digital watermarking solution to guarantee the integrity of and detect tampering with data of probative importance. Since the payload for verification is embedded in the contents, algorithms for reversible embedding and extraction, imperceptibility, payload capacity, and computational time are issues to evaluate. Thus, we propose a reversible and imperceptible audio information-hiding algorithm based on modified integer discrete cosine transform (intDCT) coefficient expansion. In this work, the original signal is segmented into fixed-length frames, and then intDCT is applied to each frame to transform signals from the time domain into integer DCT coefficients. Expansion is applied to DCT coefficients at a higher frequency to reserve hiding capacity. Objective evaluation of speech quality is conducted using listening quality objective mean opinion (MOS-LQO) and the segmental signal-to-noise ratio (segSNR). The audio quality of different frame lengths and capacities is evaluated. Averages of 4.41 for MOS-LQO and 23.314 [dB] for segSNR for 112 ITU-T test signals were obtained with a capacity of 8000 bps, which assured imperceptibility with the sufficient capacity of the proposed method. This shows comparable audio quality to conventional work based on Linear Predictive Coding (LPC) regarding MOS-LQO. However, all segSNR scores of the proposed method have comparable or better performance in the time domain. Additionally, comparing histograms of the normalized maximum absolute value of stego data shows a lower possibility of overflow than the LPC method. A computational cost, including hiding and transforming, is an average of 4.884 s to process a 10 s audio clip. Blind tampering detection without the original data is achieved by the proposed embedding and extraction method.

Many robots that play with humans have been developed so far, but developing a robot that physically contacts humans while playing is challenging. We have developed robots that play tag with humans, which find players, approach them, and move away from them. However, the developed algorithm for approaching a player was insufficient because it did not consider how the arms are attached to the robot. Therefore, in this paper, we assume that the arms are fixed on both sides of the robot and develop a new algorithm to approach the player and touch them with an arm. Since the algorithm aims to move along a circular orbit around a player, we call this algorithm “the go-round mode”. To investigate the effectiveness of the proposed method, we conducted two experiments. The first is a simulation experiment, which showed that the proposed method outperformed the previous one. In the second experiment, we implemented the proposed method in a real robot and conducted an experiment to chase and touch the player. As a result, the robot could touch the player in all the trials without collision.

Abstract Aerosols supply bioaccessible iron to marine biota which could affect climate through biogeochemical feedbacks. This paper review progresses in research on pyrogenic aerosol iron. Observations and laboratory experiments indicate that the iron solubility of pyrogenic aerosol can be considerably higher than lithogenic aerosol. Aerosol models highlight a significant contribution of pyrogenic aerosols (~20%) to the atmospheric supply of dissolved iron into the ocean. Some ocean models suggest a higher efficiency of pyrogenic iron in enhancing marine productivity than lithogenic sources. It is, however, challenging to quantitatively estimate its impact on the marine biogeochemical cycles under the changing air quality and climate.

Abstract Mineral dust is the major source of external micro-nutrients such as iron (Fe) to the open ocean. However, large uncertainties in model estimates of Fe emissions and aerosol-bearing Fe solubility (i.e., the ratio of labile Fe (LFe) to total Fe (TFe)) in the Southern Hemisphere (SH) hampered accurate estimates of atmospheric delivery of bioavailable Fe to the Southern Ocean. This study applied an inverse modeling technique to a global aerosol chemistry transport model (IMPACT) in order to optimize predictions of mineral aerosol Fe concentrations based on recent observational data over Australian coastal regions (110°E–160°E and 10°S–41°S). The optimized (a posteriori) model did not only better capture aerosol TFe concentrations downwind from Australian dust outbreak but also successfully reproduced enhanced Fe solubility (7.8 ± 8.4%) and resulted in much better agreement of LFe concentrations with the field measurements (1.4 ± 1.5 vs. 1.4 ± 2.3 ng Fe m–3). The a posteriori model estimates suggested that bushfires contributed a large fraction of LFe concentrations in aerosols, although substantial contribution from missing sources (e.g., coal mining activities, volcanic eruption, and secondary formation) was still inferred. These findings may have important implications for the projection of future micro-nutrient supply to the oceans as increasing frequency and intensity of open biomass burning are projected in the SH.

This paper describes a robotic system that supports the remote teaching of technical drawing. The aim of the system is to enable a remote class of paper-based technical drawing, where the students draw the drawing in a classroom, and the teacher gives instructions to the students from a remote place while confirming the paper drawing. The robotic system has a document camera for confirming the paper, a projector, a flat screen to project a cursor on the paper, and a video conference system for communication between the teacher and the students. We conducted two experiments. The first experiment verified the usefulness of a projected cursor. Eight participants evaluated the comprehensibility of the drawing check instruction with or without the projected cursor, and the results suggested that the use of the cursor made the instructions more comprehensible. The second experiment was conducted in a real drawing class. We asked the students in the class to answer a questionnaire to evaluate the robotic system. The result showed that the students had a good impression (useful, easy to use, and fun) of the system. The contribution of our work is twofold. First, it enables a teacher in a remote site to point to a part of the paper to enhance the interaction. Second, the developed system enabled both the students and the teacher to view the paper from their own viewpoints.

Abstract We are developing a robot that can play an outdoor game with children. In realizing such a robot, the person detection and tracking methods play an important role. In this paper, we propose methods for improving person detection and tracking using a laser range finder (LRF). A problem facing player detection using an LRF during play is that detection of a player’s body sometimes fails because of the posture of the player. Thus, we propose two improvements of the conventional person detection algorithm. One is to determine the body boundary when an arm overlaps the body, and the other is to choose the most probable object as the body among the detected objects. Another problem is how to track players who are hidden by other players. We introduce a new algorithm to robustly track a player even when the player is behind other players. We conducted three experiments. In the first experiment, we investigate the performance improvement by the proposed player detection algorithm. The second experiment proves that the proposed player-tracking algorithm can robustly track a player even after the LRF has lost that player. In the last experiment, we conduct the game Darumasan ga Koronda using the developed system and confirm that the system can perform the game without large problems.

The harlequin ladybird beetle, Harmonia axyridis, has remarkable phenotypic diversity, with over 200 colour patterns. Here, Ando et al. show that this patterning is regulated by the transcription factor gene pannier and has diversified by repeated inversions and cis-regulatory modifications of pannier.

Abstract Iron (Fe) oxides in aerosols are known to absorb sun light and heat the atmosphere. However, the radiative forcing (RF) of light-absorbing aerosols of pyrogenetic Fe oxides is ignored in climate models. For the first time, we use a global chemical transport model and a radiative transfer model to estimate the RF by light-absorbing aerosols of pyrogenetic Fe oxides. The model results suggest that strongly absorbing Fe oxides (magnetite) contribute a RF that is about 10% of the RF due to black carbon (BC) over East Asia. The seasonal average of the RF due to dark Fe-rich mineral particles over East Asia (0.4–1.0 W m−2) is comparable to that over major biomass burning regions. This additional warming effect is amplified over polluted regions where the iron and steel industries have been recently developed. These findings may have important implications for the projection of the climate change, due to the rapid growth in energy consumption of the heavy industry in newly developing countries.

The development of robots that play with humans is a challenging topic for robotics. We are developing a robot that plays tag with human players. To realize such a robot, it needs to observe the players and obstacles around it, chase a target player, and touch the player without collision. To achieve this task, we propose two methods. The first one is the player tracking method, by which the robot moves towards a virtual circle surrounding the target player. We used a laser range finder (LRF) as a sensor for player tracking. The second one is a motion control method after approaching the player. Here, the robot moves away from the player by moving towards the opposite side to the player. We conducted a simulation experiment and an experiment using a real robot. Both experiments proved that with the proposed tracking method, the robot properly chased the player and moved away from the player without collision. The contribution of this paper is the development of a robot control method to approach a human and then move away safely.

Many methods have been proposed for avoiding obstacles in robotic systems. However, a robotic system that moves without colliding with obstacles and people, while still being mentally safe to the persons nearby, has not yet been realized. In this paper, we describe the development of a method for a mobile robot to avoid a pedestrian approaching from the front and to pass him/her by while preserving the “public distance” of personal space. We assume a robot that moves along a prerecorded path. When the robot detects a pedestrian using a laser range finder (LRF), it calculates the trajectory to avoid the pedestrian considering their personal space, passes by the pedestrian, and returns to the original trajectory. We introduce a virtual target to control the robot moving along the path, such that it can use the same control strategy as for human-following behavior. We carry out experiments to evaluate the method along three routes, in which the robot functioned without problems. The distance between the robot and the pedestrian was 9.3 m, on average, when the robot started to use avoiding behavior, which is large enough to keep a public distance from a pedestrian. When the robot passed by the pedestrian, the minimum distance between them was 1.19 m, which was large enough for passing safely.

In this paper, we describe the development of a robot that plays the “darumasan-ga- koronda” game with human players. It is a big challenge to develop a robot that can play with human players. We first observed the human−human game play and listed the problems for a robot to play the game with human players. We pointed out the following six tasks: detect players, track the detected players, decide if players are freezing (the “out” judgment), move to the nearest player, touch the player, and move back to the original position. Then, we designed a robot that uses a laser range finder (LRF) to measure the players and a robot avatar to interact with the players. Then, we developed the components needed to play the game. In this paper, we describe the details of the development of the arm to touch the player, the “out” judgment, and a function to return to the origin after finishing the game. Then, we describe an experimental result of the evaluation of the total system, as well as the track record of the demonstrations in the events.

A life-support service robot must avoid both static and dynamic obstacles for working in a real environment. Here, a static obstacle means an obstacle that does not move, and a dynamic obstacle is the one that moves. Assuming the robot is following a target person, we discuss how the robot avoids a crowd through which the target person passes and arrives at the target position. The purpose of this paper is to propose a crowd avoidance method that makes a robot to be able to avoid both static and dynamic obstacles. The method uses the surface points of the obstacles to form an avoidance region, and the robot moves along the edge of the region. We conducted experiments assuming various situations such that the robot was blocked, there was a wide gap in the crowd, or a person in the crowd yielded for the robot to pass through. As an experimental result, it was confirmed the robot could avoid the crowd even when the obstacles were aligned in an “inverted wedge” shape.

Prosody plays an important role in speech communication between humans. Although several computer-assisted language learning (CALL) systems with utterance evaluation function have been developed, the accuracy of their prosody evaluation is still poor. In the present paper, we develop new methods by which to evaluate the rhythm and intonation of English sentences uttered by Japanese learners. The novel features of our study are as follows: (1) new prosodic features are added to traditional features, and (2) word importance factors are introduced in the calculation of intonation score. The word importance factor is automatically estimated using the ordinary least squares method and is optimized based on word clusters generated by a decision tree. Experiments conducted herein reveal the correlation coefficient (±1.0 denotes the best correlation) between the rhythm score given by native speakers and the system was -0.55. In contrast, a conventional feature (pause insertion error rate) gave a correlation coefficient of only -0.11. The correlation coefficient between the intonation scores given by native speakers and the system was only -0.29. However, the word importance factor with decision tree clustering improved the correlation coefficient to 0.45. In addition, we propose a method of integrating the rhythm score with the intonation score, which improved the correlation coefficient from 0.45 to 0.48 for evaluating intonation.

This paper describes a method for estimating the internal state of a user of a spoken dialog system before his/her first input utterance. When actually using a dialog-based system, the user is often perplexed by the prompt. A typical system provides more detailed information to a user who is taking time to make an input utterance, but such assistance is nuisance if the user is merely considering how to answer the prompt. To respond appropriately, the spoken dialog system should be able to consider the user’s internal state before the user’s input. Conventional studies on user modeling have focused on the linguistic information of the utterance for estimating the user’s internal state, but this approach cannot estimate the user’s state until the end of the user’s first utterance. Therefore, we focused on the user’s nonverbal output such as fillers, silence, or head-moving until the beginning of the input utterance. The experimental data was collected on a Wizard of Oz basis, and the labels were decided by five evaluators. Finally, we conducted a discrimination experiment with the trained user model using combined features. As a three-class discrimination result, we obtained about 85% accuracy in an open test.

We are developing a method of Web-based unsupervised language model adaptation for recognition of spoken documents. The proposed method chooses keywords from the preliminary recognition result and retrieves Web documents using the chosen keywords. A problem is that the selected keywords tend to contain misrecognized words. The proposed method introduces two new ideas for avoiding the effects of keywords derived from misrecognized words. The first idea is to compose multiple queries from selected keyword candidates so that the misrecognized words and correct words do not fall into one query. The second idea is that the number of Web documents downloaded for each query is determined according to the “query relevance.” Combining these two ideas, we can alleviate bad effect of misrecognized keywords by decreasing the number of downloaded Web documents from queries that contain misrecognized keywords. Finally, we examine a method of determining the number of iterative adaptations based on the recognition likelihood. Experiments have shown that the proposed stopping criterion can determine almost the optimum number of iterations. In the final experiment, the word accuracy without adaptation (55.29%) was improved to 60.38%, which was 1.13 point better than the result of the conventional unsupervised adaptation method (59.25%).

Recently, several music information retrieval (MIR) systems which retrieve musical pieces by the user's singing voice have been developed. All of these systems use only melody information for retrieval, although lyrics information is also useful for retrieval. In this paper, we propose a new MIR system that uses both lyrics and melody information. First, we propose a new lyrics recognition method. A finite state automaton (FSA) is used as recognition grammar, and about 86% retrieval accuracy was obtained. We also develop an algorithm for verifying a hypothesis output by a lyrics recognizer. Melody information is extracted from an input song using several pieces of information of the hypothesis, and a total score is calculated from the recognition score and the verification score. From the experimental results, 95.0% retrieval accuracy was obtained with a query consisting of five words.