Search

In this study, we have developed a non-destructive inspection system for detecting external wall tile delamination by incorporating a pulse water jet impingement force into a suspended apparatus. The delamination inspection using a pulse water jet offers flexibility in adapting to variations in the distance between the inspection device and the inspection surface, a challenge inherent to the suspended apparatus. Furthermore, the delamination inspection using a pulse water jet exhibits exceptional safety and cost-effectiveness. Using this inspection system, we conducted a delamination detection test on tiles with delaminated mortar and were able to detect it. The inspection speed of this apparatus is approximately 10 centimeters per second in horizontal movement, allowing for an inspection rate of roughly 18 square meters per hour for tiles with a 50-millimeter spacing. This inspection speed is comparable to that achievable by human operators. Moreover, the system can perform inspections at heights of up to 30 meters above ground level. This inspection system has the potential to address the issue of labor shortage by facilitating automation.

Grasping and manipulating transparent objects with a robot is a challenge in robot vision. To successfully perform robotic grasping, 6D object pose estimation is needed. However, transparent objects are difficult to recognize because their appearance varies depending on the background, and modern 3D sensors cannot collect reliable depth data on transparent object surfaces due to the translucent, refractive, and specular surfaces. To address these challenges, we proposed a 6D pose estimation of transparent objects for manipulation. Given a single RGB image of transparent objects, the 2D keypoints are estimated using a deep neural network. Then, the PnP algorithm takes camera intrinsics, object model size, and keypoints as inputs to estimate the 6D pose of the object. Finally, the predicted poses of the transparent object were used for grasp planning. Our experiments demonstrated that our picking system is capable of grasping transparent objects from different backgrounds. To the best of our knowledge, this is the first time a robot has grasped transparent objects from a single RGB image. Furthermore, the experiments show that our method is better than the 6D pose estimation baselines and can be generalized to real-world images.

Transparent objects are one of the most common objects in everyday life. Estimating pose of these objects are required to pick and manipulate such objects. However, due to the absorption and refraction of light, it is hard to capture depth im- age of transparent object. In this paper, we address this problem using synthetic dataset to train deep neural network and estimate pose of known transparent objects. Synthetic dataset contains depth map of transparent object which we created in realistic looking environment. Also combining domain randomized and photorealistic images, we create desired amount of annotated data in order to network operate successfully against real world data. We conducted experiment on 3D printed transparent objects in the real environment. For future work, we are planning to build random bin picking system for transparent object.

Kumamoto Castle, consisting some towers and the stone walls called ISHIGAKI, was severely damaged by Kumamoto earthquake in 2016. As Kumamoto Castle has some important cultural properties of Japan, prompt and accurate restoration works for the damaged stone walls and the castle are needed. For the restoration works, it was difficult to estimate all collapsed stone locations since the shapes of the stones are similar. For this problem, the estimation method of possible stones based on iterative closest point algorithm using photographic imagery has been studied. This algorithm showed a good result, however, about 20% of collapsed stones were not decided. In recent years, the utilization of stone’s 3D data has been studied to recognize of the possible stones precisely. For an efficient renovation, it is needed to measure 3D data automatically in a short period of time. In this paper, the scanning system of the collapsed stones controlled by a compact control moment gyroscope (CMG) crane and an elevator to move 3D scanner is proposed. Parameter change of CMG crane which caused by the stone shape and size is suppressed by the model error compensator (MEC). The CMG crane rotates the stone around according to the up-and-down motion of the 3D scanner moved by an elevator. The effectiveness of the proposed system to archiving 3D data is evaluated by scanning dummy stones experiments.

When learning to play a musical instrument, it is important to improve the quality of self-practice. Many systems have been developed to assist practice. Some practice assistance systems use special sensors (pressure, flow, and motion sensors) to acquire the control parameters of the musical instrument, and provide specific guidance. However, it is difficult to acquire the control parameters of wind instruments (e.g., saxophone or flute) such as flow and angle between the player and the musical instrument, since it is not possible to place sensors into the mouth. In this paper, we propose a sensorless control parameter estimation system based on the recorded sound of a wind instrument using only machine learning. In the machine learning framework, many training samples that have both sound and correct labels are required. Therefore, we generated training samples using a robotic performer. This has two advantages: (1) it is easy to obtain many training samples with exhaustive control parameters, and (2) we can use the correct labels as the given control parameters of the robot. In addition to the samples generated by the robot, some human performance data were also used for training to construct an estimation model that enhanced the feature differences between robot and human performance. Finally, a flute control parameter estimation system was developed, and its estimation accuracy for eight novice flute players was evaluated using the Spearman’s rank correlation coefficient. The experimental results showed that the proposed system was able to estimate human control parameters with high accuracy.

In recent years, many researchers have shown increasing interest in music information retrieval (MIR) applications, with automatic chord recognition being one of the popular tasks. Many studies have achieved/demonstrated considerable improvement using deep learning based models in automatic chord recognition problems. However, most of the existing models have focused on simple chord recognition, which classifies the root note with the major, minor, and seventh chords. Furthermore, in learning-based recognition, it is critical to collect high-quality and large amounts of training data to achieve the desired performance. In this paper, we present a multi-task learning (MTL) model for a guitar chord recognition task, where the model is trained using a relatively large-vocabulary guitar chord dataset. To solve data scarcity issues, a physical data augmentation method that directly records the chord dataset from a robotic performer is employed. Deep learning based MTL is proposed to improve the performance of automatic chord recognition with the proposed physical data augmentation dataset. The proposed MTL model is compared with four baseline models and its corresponding single-task learning model using two types of datasets, including a human dataset and a human combined with the augmented dataset. The proposed methods outperform the baseline models, and the results show that most scores of the proposed multi-task learning model are better than those of the corresponding single-task learning model. The experimental results demonstrate that physical data augmentation is an effective method for increasing the dataset size for guitar chord recognition tasks.