Search

This study delves into verifying the applicability of object co-occurrence information generated by a large-scale language model (LLM) to enhance a robot’s spatial ability to understand objects in the real world. Co-occurrence information is crucial in enabling robots to perceive and navigate their surroundings. LLM can generate object co-occurrence information based on the learned representations acquired from the learning process. However, the challenge lies in determining whether the co-occurrence gleaned from linguistic data can effectively translate to real-world object relationships, a concept yet to be thoroughly examined. After providing category information about a specific situation, this paper compares and evaluates the co-occurrence degree (co-occurrence coefficient) output by gpt-4-turbo-2024-04-09 (GPT-4) against the object pair data from the ScanNet v2 dataset. The results revealed that GPT-4 achieved a high recall of 0.78 across various situation categories annotated by ScanNet v2, although its precision was relatively low at an average of 0.29. The root mean square error of the co-occurrence coefficient was 0.31. While GPT-4 tends to output slightly higher co-occurrence coefficients, it effectively captures the overall co-occurrence patterns observed in the ScanNet v2 dataset. GPT-4 produced co-occurrence information for more objects than those available in ScanNet v2 while covering co-occurrences among objects within ScanNet v2. These results demonstrate that integrating co-occurrence data from different sources could enhance the ability to recognize real-world objects and potentially strengthen robot intelligence.

In a fire outbreak, firefighters are expected to rapidly extinguish fires to stop the spread of damage and prevent secondary disasters. We proposed the concept of a dragon firefighter (DFF), which is a flying-hose-type firefighting robot. We developed a 3.6 m long DFF equipped with two nozzle units and achieved stable flight. However, the system was not yet completed because the root of the robot, which should have been operated remotely, was operated manually. In addition, the system’s reliability was insufficient to successfully repeat the demonstration several times. The development of a robot demonstration system is crucial for the practical application of such a firefighting robot. In this study, we developed a demonstration system for a remotely controllable 4 m flying firehose robot for demonstration at the World Robot Summit 2020 (WRS 2020) opening ceremony in Fukushima as a milestone. This paper focuses on the following issues: 1): installation of the remotely controllable mobile base, 2): redesign of the water channels (the sizes of nozzle outlets) to get enough thrusts to fly with a fire engine, 3): development of nozzle units with a larger movable range (1.5 times larger than the conventional nozzle) in addition to waterproofing technique to improve system reliability, and 4): redesign of a passive damping mechanism to ensure better stability. Thus, a firefighting demonstration was successfully conducted at the opening ceremony of the World Robot Summit 2020 in Fukushima, Japan, and we discuss the lessons learned through the demonstration. We found that the developed DFF system incorporating a mobile base could achieve remote fire extinguishing.

Abstract Continuum robots can enter narrow spaces and are useful for search and rescue missions in disaster sites. The exploration efficiency at disaster sites improves if the robots can simultaneously acquire several pieces of information. However, a continuum robot that can simultaneously acquire information to such an extent has not yet been designed. This is because attaching multiple sensors to the robot without compromising its body flexibility is challenging. In this study, we installed multiple small sensors in a distributed manner to develop a continuum-robot system with multiple information-gathering functions. In addition, a field experiment with the robot demonstrated that the gathered multiple information has a potential to improve the searching efficiency. Concretely, we developed an active scope camera with sensory functions, which was equipped with a total of 80 distributed sensors, such as inertial measurement units, microphones, speakers, and vibration sensors. Herein, we consider space-saving, noise reduction, and the ease of maintenance for designing the robot. The developed robot can communicate with all the attached sensors even if it is bent with a minimum bending radius of 250 mm. We also developed an operation interface that integrates search-support technologies using the information gathered via sensors. We demonstrated the survivor search procedure in a simulated rubble environment of the Fukushima Robot Test Field. We confirmed that the information provided through the operation interface is useful for searching and finding survivors. The limitations of the designed system are also discussed. The development of such a continuum robot system, with a great potential for several applications, extends the application of continuum robots to disaster management and will benefit the community at large.

Flexible continuum robots have considerable potential for use in exploring intricate spaces, and their ability to make large body shape deformations can increase the inspection area. We previously proposed a jet-actuated flying continuum robot for extinguishing fires. The main challenge in implementing large shape deformations is accommodating the twisting of the body that results from the deformation. To address this problem, we proposed a two-dimensional passive rotating nozzle unit that can expand the directionality of net force against torsion; however, it has not yet been tested on a flying robot. In this study, we achieved the large shape deformations of a jet-actuated flying continuum robot using an improved passive rotating nozzle unit that can handle three-dimensional (3D) force. First, we developed a model of the improved nozzle unit and confirmed that the unit can increase the net force direction. Herein, the design strategy for the rotary damper to handle the instability that arises from motor limitations is discussed. The stabilized flight controller was applied to a continuum robot with the nozzle unit. Simulation results showed that the 2 m robot could perform large head bends (from 0° to 135°). Although the previous fixed nozzle unit twisted by approximately 40°, which made the extra movable range of the nozzles effectively zero, the proposed nozzle unit maintained the movable range to avoid twisting. We experimentally confirmed that the nozzle unit can expand the direction of the 3D net force, and that a large shape bending of approximately ±90° can be achieved using a 1.6 m flying robot.

Fluid jet actuation is a potential actuation technique for continuum robots. It can generate and rapidly control a relatively large force using a small and lightweight structure because a significant amount of energy can be transported through its internal channels. Recently, jet-actuated flying continuum robots have been developed using this advantageous characteristic. However, a challenging issue in controlling the robot is the fluid structure interaction between the flexible body and the internal flowing fluid. This interaction often causes instability in the pipe conveying fluid. In this study, as a first step to address this issue, we propose a stabilized controller (vertical position control) for a jet-actuated two-dimensional cantilevered pipe with a nozzle unit at the tip using the damping effect of the internal flowing fluid and verify the controller with a real robot. Specifically, a model is constructed with the net force of the jets as the control input. A simple controller that can constantly decrease the energy function is proposed by utilizing the damping effect of the flowing fluid. Numerical simulations verify the stability of the system regardless of the flow velocity. In particular, fluid damping mainly suppresses the higher-order mode oscillations. Moreover, the stability of the system can be improved by adjusting the controller gains. We also conduct experiments using an actual robot to verify the simulation results. The vibrations can be damped by the fluid effect, and the stability can be improved using the proposed controller.

Abstract Generation of stable and realistic haptic feedback during mid-air gesture interactions have recently garnered significant research interest. However, the limitations of the sensing technologies such as unstable tracking, range limitations, nonuniform sampling duration, self occlusions, and motion recognition faults significantly distort motion based haptic feedback to a large extent. In this paper, we propose and implement a hidden Markov model (HMM)-based motion synthesis method to generate stable concurrent and terminal vibrotactile feedback. The system tracks human gestures during interaction and recreates smooth, synchronized motion data from detected HMM states. Four gestures—tapping, three-fingered zooming, vertical dragging, and horizontal dragging—were used in the study to evaluate the performance of the motion synthesis methodology. The reference motion curves and corresponding primitive motion elements to be synthesized for each gesture were obtained from multiple subjects at different interaction speeds by using a stable motion tracking sensor. Both objective and subjective evaluations were conducted to evaluate the performance of the motion synthesis model in controlling both concurrent and terminal vibrotactile feedback. Objective evaluation shows that synthesized motion data had a high correlation for shape and end-timings with the reference motion data compared to measured and moving average filtered data. The mean $$R^{2}$$ R2 values for synthesized motion data was always greater than 0.7 even under unstable tracking conditions. The experimental results of subjective evaluation from nine subjects showed significant improvement in perceived synchronization of vibrotactile feedback based on synthesized motion.

Humans perceive and discriminate high-frequency tactile vibrations based on the intensity and envelope of the stimuli. However, no studies have investigated exactly how the envelope and intensity each affect the ability to discriminate. The objectives of this paper are to identify the boundary at which the envelope begins not to strongly affect the ability to discriminate vibrations and to investigate the effects of the carrier frequency and intensity on the discrimination ability. The results of our testing showed that the ability to discriminate was dependent on the envelope frequency and that the ability to discriminate sinusoidal and amplitude-modulated (AM) vibrations in which the envelope frequency ranged from 12 to 50 Hz was higher than that required to discriminate sinusoidal and AM vibrations in which the envelope frequency was above 80 Hz. When the envelope frequency of an AM vibration was 125 Hz, the ability to discriminate sinusoidal and AM vibrations was found to be low and no significant difference was noted in comparison to discriminating AM vibrations with the same envelope frequency. These results suggest that the boundary for envelope perception was at an envelope frequency of around 80-125 Hz at low intensities and that the carrier frequency had little effect on the discrimination, although the discrimination ability tended to increase as the intensity increased.

A human can judge the hardness of an object based on the damped natural vibration caused by tapping the surface of the object using a fingertip. In this study, we investigated the influence of the dynamic characteristics of vibrations on the hardness perceived by tapping. Subjectively reported hardness values were related to the dynamic stiffness of several objects. The dynamic stiffness, which characterizes the impulsive response of an object, was acquired across the 40–1,000 Hz frequency range for cuboids of 14 types of materials by administering a hammering test. We performed two psychophysical experiments—a ranking task and a magnitude-estimation tasks—wherein participants rated the perceived hardness of each block by tapping it with a finger. We found that the perceptual effect of dynamic stiffness depends on the frequency. Its effect displayed a peak around 300 Hz and decreased or disappeared at higher frequencies, at which human perceptual capabilities are limited. The acquired results help design hardness experienced by products.