Search

In this article, we present a gliding efficiency optimization strategy based on deep reinforcement learning for a gliding robotic fish. For the gliding motion in shallow waters, the nonsteady motion strongly impacts the gliding range and also reduces efficiency. This article presents a concept of transient gliding motion and illustrates its importance for the gliding robotic fish. For better gliding performance of active fins, several pectoral fins with different sizes are designed and their hydrodynamics and optimizing capabilities are analyzed by computational fluid dynamics simulation. Then, a double deep Q network-based optimization strategy is proposed to improve gliding efficiency by active pectoral fins, in which an adversarial model and a two-stage reward function are presented for the adequate calculation of gliding range. Simulations are conducted to validate the convergence and effectiveness of the proposed strategy. The aquatic experiments are carried out to further verify the proposed strategy. The results reveal that the optimization strategy can save about 4.88% of energy and 19.45% of travel time. This article provides clues to the design of active control surfaces and improvement of gliding efficiency for underwater vehicles. Remarkably, the proposed strategy can significantly improve the duration and endow the robot with the potential to perform complex tasks. [ABSTRACT FROM AUTHOR]

This article proposes the mechanical design and dynamic model of an innovative manta-inspired robot system for both efficient fast swimming and high spatial maneuverability. Inspired by some biological studies, a pair of unique pectoral fins with six separate degrees of freedoms (DOFs) are developed. The novel design is characterized by an improved crank-rocker mechanism and a distinctive horizontal DOF. The former not only endows the robot with high swimming speed, but also guarantees efficient flapping patterns which are close to manta rays. The latter is employed to coordinate with the flapping movement, allowing remarkable pitch adjustment. Further, the basic motion strategy is presented by detailed analyses to the pectoral fins. Besides, based on the Morrison equation and infinitesimal method, a complete dynamic model for robotic manta with flexible pectoral fins is established, whose parameters are determined through experimental data. Moreover, the linear swimming and pitching experiments are conducted, demonstrating the prominent movement performance of the presented design and the effectiveness of the dynamic model. The obtained results shed light on updated design and control of next-generation agile underwater vehicles and robots capable of multimodal motions in dynamic and complex aquatic environments. [ABSTRACT FROM AUTHOR]

Implementing underwater target tracking remains difficult for a free-swimming robotic fish owing to the intrinsically reciprocating motion in fishlike propulsion. In this article, we present a novel robotic fish platform with a camera stabilizing system and achieve real-time two-dimensional target tracking assisted by reinforcement learning (RL) in continuous environments. More specifically, we first develop an active visual tracking system based on cascade control structure to obtain the relative orientation between the robotic fish and the underwater target. Then, we propose a target tracking controller dealing with continuous state and action spaces based on deep RL (DRL). The controller takes the position of the target object as input and yields the motion parameters of the bioinspired central pattern generator governed robotic fish. The robustness and adaptability of the proposed controller as well as the influence of time-delays on the control system are explored via simulated experiments under different scenarios. Finally, both static and dynamic tracking experiments on the actual robotic fish demonstrate the effectiveness of the proposed mechatronic design and control methods, providing insights to executing aquatic vision-based tracking tasks. [ABSTRACT FROM AUTHOR]

In this article, a robot system for intelligent water surface cleaner named IWSCR is developed to collect floating plastic garbage. It is able to accomplish three major tasks autonomously, i.e., cruise and detection, tracking and steering, and grasping and collection. The challenges behind these tasks involve how to realize the accurate and real-time garbage detection, how to resist the disturbances while IWSCR conducts vision-based steering, and how to grasp the floating garbage reliably despite the turbulent conditions on the surface of the water. To overcome these difficulties, three key techniques are proposed for IWSCR. First, the YOLOv3 network, which is widely applied in the high speed and accuracy object detection field, is trained on the proposed floating garbage dataset to realize accurate and real-time garbage detection. Next, to improve the ability of resisting disturbances, a control law based on the sliding-mode controller is proposed for vision-based steering. Furthermore, inspired by the stability of floating bottles in fluid, a feasible grasping strategy is utilized for IWSCR. Finally, the experimental results demonstrate that IWSCR is competent to carry out the task of water surface cleaning. [ABSTRACT FROM AUTHOR]

In this article, we propose an efficient locomotion control method for a two-dimensional tracking task of a biomimetic four-joint robotic fish. Regarding this issue as a comprehensive optimization procedure, we propose an optimization-based cooperative structured control framework, in which the combination of evolutionary strategy and deep deterministic policy gradient is employed to optimize the same objective function. An inconsistent optimization method is presented to further enhance the effect of parameter optimization on central pattern generator model. Moreover, for the sake of a higher reward and better robustness of controllers governed by deep reinforcement learning, we propose a linear weighted controller trained with periodic method. Extensive simulation and experimental results verify the significant energy saving of the proposed method in tracking tasks. Noticeably, the cooperative structured control can save $\text{23.97}\%$ , $\text{22.13}\%$ , and $\text{38.72}\%$ energy compared with sliding mode control, active disturbance rejection control, and proportional-integral-differential control in experiments, respectively, holding a great promise for the long-term intelligent work of the biomimetic robotic fish in complex aquatic environments. [ABSTRACT FROM AUTHOR]

This article proposes a model predictive control (MPC)-based depth control system for the gliding motion of a gliding robotic dolphin. An injector-based buoyancy-driven mechanism is employed to achieve more precise control of net buoyancy. In the system, a novel framework of depth control is proposed on the basis of a simplified model, including a depth controller with improved MPC, a heading controller with velocity-based proportional-integral-derivative, and a sliding mode observer. Extensive simulation and experimental results demonstrate the effectiveness of the proposed control methods. In particular, a variety of slider-based experiments are also conducted to explore the performance of a movable slider in the depth control so as to better govern the gliding angle. The results obtained reveal that it is feasible to realize regular gliding angles via regulating the slider, which offers promising prospects for bio-inspired gliding robots playing a key role in ocean exploration. [ABSTRACT FROM AUTHOR]

In this article, an extended state observer (ESO)-based controller with a model predictive governor is designed for 3-D trajectory tracking of underactuated underwater vehicles. The proposed control scheme takes three primary challenges including underactuated property, velocity constraint, and lumped disturbance into consideration. With respect to the model predictive governor, an underactuated kinematic tracking error model is utilized to produce reference velocities. Meanwhile, a heading angle compensation mechanism is utilized to avoid the steady tracking errors resulting from dynamics coupling of the vehicle. Besides, an ESO is designed to estimate the lumped disturbances and unmeasured velocity states. Based on the ESO, a kinetic controller is offered to accomplish the precise velocity tracking only in virtue of the position and orientation information. Note that this article details both the design process of the control scheme and rigorous theoretical analysis. Eventually, simulation and experimental results demonstrate the feasibility and superiority of the proposed method. Notably, this work lays the foundation for the underactuated trajectory tracking control in complicated and turbulent underwater environments. [ABSTRACT FROM AUTHOR]

This article proposes a modified rapidly exploring random tree (RRT)-based path planner and behavior-based cooperative tracking strategy for a dual robotic dolphin system to fulfill a cooperative target-tracking task. Specifically, with full consideration of both task requirements and mechatronic configuration, a robotic dolphin with a waist-caudal propulsive mechanism for thrust forces and differential bilateral flippers for maneuverability is developed. To satisfy the demand of fast path planning, a variant RRT algorithm is employed to generate feasible paths for the dual robotic dolphins with fewer waypoints, faster convergence speed, and better stability. Furthermore, a behavior-based approach in conjunction with centralized architecture is implemented to achieve high-level decision-making. Finally, simulations, analysis, as well as field experiments are carried out to verify the effectiveness of the proposed control scheme. The success of the experiments further offers insight into the mechanisms of cooperative multirobot target tracking in aquatic environments. [ABSTRACT FROM AUTHOR]

This article presents the mechanical design and locomotion control of a novel tuna-inspired robot system for both fast swimming and high maneuverability. Mechanically, the developed robotic fish named CasiTuna comprises three important parts, i.e., an innovative two-motor-actuated propulsive mechanism, a buoyancy adjustment structure, and a pair of pectoral fins. Unlike most robotic fishes’ multiple concatenated links-based propulsive mechanism, CasiTuna’s two-motor-actuated one places both motors in the anterior body and utilizes a transmission system to achieve tuna-like lateral undulations. Meanwhile, the buoyancy adjustment mechanism in conjunction with pectoral fins endows the robot with the capability of three-dimensional maneuverability. Kinematic and dynamic analyses are further conducted to reveal the interactive hydrodynamic forces. Regarding the locomotion control method, a bio-inspired central pattern generator-based controller is adopted to achieve multimodal swimming. In particular, two kinds of turning maneuvers are implemented and discussed. Aquatic experiments, including straight swimming, circular turning, and nearly static pitching validate the effectiveness of proposed mechatronic design and locomotion control methods. Remarkably, CasiTuna achieved a peak forward speed of 0.8 m/s (corresponding to 1.52 body lengths per second) and a minimum turning radius of less than 0.3 body lengths. [ABSTRACT FROM AUTHOR]

To improve the operation performance of underwater robots, a novel micro cable-driven pan-tilt camera system is developed, which can play an important role in underwater environmental perception and visual measurement. The designed camera system encompasses three parts: a miniature mechanical structure, a real-time target tracking and location module, and an underwater spherical refraction calibration module. Originally, a cable-driven model is adopted to establish a micro and flexible pan-tilt structure, for which the corresponding active target tracking and location algorithms are proposed. More importantly, for eliminating the measurement error caused by the underwater spherical refraction, an advanced nondominated sorting genetic algorithm (NSGA-II)-based underwater calibration algorithm is presented with the aid of the comprehensive geometric spherical refraction model. By integrating the flexible pan-tilt camera and the proposed calibration algorithm, the performance of underwater environmental perception and visual measurement can be greatly improved. Finally, extensive experiments in air and underwater scenarios are conducted. According to the utilized evaluation index, the deviation of target tracking and underwater location errors can be maintained within 0.06 rad and 21 mm, respectively. The proposed vision system sheds light on precise perception and interaction tasks of underwater vehicles. [ABSTRACT FROM AUTHOR]

This paper presents a three-dimensional (3-D) path planning method that combines the gliding with dolphin-like motions for the gliding robotic dolphin. A specific task that the robot uses the gliding motion for long-distance cruise and the dolphin-like motion for maneuverable obstacle avoidance is employed. The results of simulations and aquatic experiments validate the full-state dynamic model and the specific task, further offer some theoretical supports for 3-D path planning. Further, the 3-D path planning method is composed of three main components: 1) gliding path generation; 2) improved Astar (A*) algorithm; and 3) segmented Bezier curve smoothing. First, the gliding path is generated autonomously with the kinematic constraints that are obtained via the simulations of dynamic model. Furthermore, when the obstacles are detected by the sonar, an improved A* algorithm is employed to avoid the obstacles. Afterward, considering the path planned by A* is unsmoothed, a segmented Bezier curve method is presented. Simulation results demonstrate the effectiveness of the method, offering valuable insight into the utilization of hybrid underwater robots in the context of real-time task execution. [ABSTRACT FROM AUTHOR]

This paper proposes a novel hierarchical framework to achieve real-time path planning and following for a gliding robotic dolphin, including a learning-based path planner and an adaptive following controller. Subsequent to considering higher intelligence in unknown ocean environment, we present a novel hierarchical deep Q-network method to separately plan the collision avoidance path and the approach path, and also design different continuous-states under the kinematic constraints. Next, we adopt an improved line-of-sight method to obtain the desired points from planned path. More importantly, we derive a nonlinear control law based on backstepping technique, and specially avoid singularities in the law derivation using barrier Lyapunov function. In particular, the unknown model parameters are adaptively calculated to make the controller more robust. Finally, extensive simulation and experimental results verify the effectiveness of the proposed methods, providing a new idea to further ocean exploration. [ABSTRACT FROM AUTHOR]

Object detection has been vigorously investigated for years but fast accurate detection for real-world scenes remains a very challenging problem. Overcoming drawbacks of single-stage detectors, we take aim at precisely detecting objects for static and temporal scenes in real time. Firstly, as a dual refinement mechanism, a novel anchor-offset detection is designed, which includes an anchor refinement, a feature location refinement, and a deformable detection head. This new detection mode is able to simultaneously perform two-step regression and capture accurate object features. Based on the anchor-offset detection, a dual refinement network (DRNet) is developed for high-performance static detection, where a multi-deformable head is further designed to leverage contextual information for describing objects. As for temporal detection in videos, temporal refinement networks (TRNet) and temporal dual refinement networks (TDRNet) are developed by propagating the refinement information across time. We also propose a soft refinement strategy to temporally match object motion with the previous refinement. Our proposed methods are evaluated on PASCAL VOC, COCO, and ImageNet VID datasets. Extensive comparisons on static and temporal detection verify the superiority of DRNet, TRNet, and TDRNet. Consequently, our developed approaches run in a fairly fast speed, and in the meantime achieve a significantly enhanced detection accuracy, i.e., 84.4% mAP on VOC 2007, 83.6% mAP on VOC 2012, 69.4% mAP on VID 2017, and 42.4% AP on COCO. Ultimately, producing encouraging results, our methods are applied to online underwater object detection and grasping with an autonomous system. Codes are publicly available at https://github.com/SeanChenxy/TDRN. [ABSTRACT FROM AUTHOR]

This article proposes a miniature underwater angle-of-attack (AoA) sensor appropriate to self-propelled fin-actuated underwater robots, featuring real-time measurement of AoA relative to the incoming flow. Specifically, the developed sensor adopts a vane-rotating structure with a high-performance Hall chip that is responsible for detection of the angular deviation. Meanwhile, the Hall-effect-based acquisition scheme endows the device with several features, including low rotational damping, to satisfy underwater usage requirements. Another contribution is that an AoA model is derived to allow the real-time acquisition of AoA at the trailing edge of the caudal fin assumed in a quiescent flow. Based on this model, the AoA profile is obtained as theoretical reference with only the control laws required. Finally, the performance of the numerical model and the developed sensor are investigated through aquatic experiments assisted by a global visual measurement system. The experimental results demonstrate that the acquired AoA possesses over 0.97 Parson correlation coefficient with the theoretical data, verifying the effectiveness of the proposed mechatronic design. The developed AoA sensor and its application to robotic fish could provide potential assistance for the flow control and hydrodynamic optimization of fin-actuated underwater robots. [ABSTRACT FROM AUTHOR]

The intrinsically reciprocating motion in fishlike propulsion causes severe attitude instability of a robotic fish, which poses enormous challenges for environmental perception and autonomous operation. To address this issue, in this article, we propose a reaction-wheel-based control framework for guaranteeing the roll stability of the robotic fish. The mechatronic design and dynamic model of the designed robotic fish with an internal rotor are presented. By means of the simplified model and frequency domain analysis, the effect factors about roll stability are concretely analyzed. More importantly, a hybrid controller that combines a sliding mode controller with a neural network feedforward compensator is developed to reject the severe disturbance on roll angle. Then, the Lyapunov stability theory is utilized to analyze the stability and convergence property of the closed-loop system. Finally, the experimental results show that the proposed methods possess more significant performances than the passive stabilization method, which provides a valuable reference for attitude stabilization control and robust environmental perception of underwater robots. [ABSTRACT FROM AUTHOR]

Most existing robotic fish have a large body size driven by servo motor system, while conventional small-sized actuators hardly generate a high swimming performance. This paper reports a miniature untethered robotic fish, whose body length is 69 mm. In particular, a newly designed magnetic actuator system (MAS) is equipped, which guarantees both small-sized dimension and flexibility of the robot. More specifically, the magnetic field generated by a permanent magnet is first investigated based on Biot–Savart law. Then, a novel tail-beating rhythm called magnetically actuated pulse width modulation (MAPWM) is modeled for the new actuator system. Further, an MAPWM-based control method is presented, in which the duty ratio of MAPWAM is innovatively utilized to realize the turning maneuvers for the first time. In addition, Lagrangian method is employed to establish the dynamic model to assess the MAPWM-based control method and the turning performance of the robotic fish. To further improve the maneuverability, the effect of a shape-variable caudal fin is analyzed based on computational fluid dynamics and the built dynamic model. Finally, combined with the MAS, the MAPWM-based control method, and the optimally selected caudal fin, extensive aquatic experiments are conducted on the robotic prototype. The results indicate that the developed miniature robotic fish achieves a considerably higher level of maneuverability in terms of turning radius when compared to swimming robots with equivalent dimensions. [ABSTRACT FROM AUTHOR]

Temporal object detection has attracted significant attention, but most popular detection methods cannot leverage rich temporal information in videos. Very recently, many algorithms have been developed for video detection task, yet very few approaches can achieve real-time online object detection in videos. In this paper, based on the attention mechanism and convolutional long short-term memory (ConvLSTM), we propose a temporal single-shot detector (TSSD) for real-world detection. Distinct from the previous methods, we take aim at temporally integrating pyramidal feature hierarchy using ConvLSTM, and design a novel structure, including a low-level temporal unit as well as a high-level one for multiscale feature maps. Moreover, we develop a creative temporal analysis unit, namely, attentional ConvLSTM, in which a temporal attention mechanism is specially tailored for background suppression and scale suppression, while a ConvLSTM integrates attention-aware features across time. An association loss and a multistep training are designed for temporal coherence. Besides, an online tubelet analysis (OTA) is exploited for identification. Our framework is evaluated on ImageNet VID dataset and 2DMOT15 dataset. Extensive comparisons on the detection and tracking capability validate the superiority of the proposed approach. Consequently, the developed TSSD-OTA achieves a fast speed and an overall competitive performance in terms of detection and tracking. Finally, a real-world maneuver is conducted for underwater object grasping. [ABSTRACT FROM AUTHOR]

This paper investigates a controllable fluke's regulation effect on the gliding motion of a self-propelled robotic dolphin in both theory and practice. In theory, based on a dynamic model, the gliding equilibria is first analyzed. Through deriving the explicit expressions of the pitch angle, gliding angle, and velocities, we explore how the deflection angle of the controllable fluke changes the gliding performance. Then, aided by these theoretical analyses, a pitch control strategy relying on the fluke is constructed. In simulations, the robotic dolphin is able to track a given pitch angle during gliding, directly through adopting this control strategy to regulate the fluke's deflection angle. Furthermore, extensive gliding experiments in different fluke's deflection angles are carried out. From the statistical data, we further explore the remarkable effects of the controllable fluke on the gliding performance in practice. Besides, the other pitch control experiment is also executed to validate the effectiveness of the proposed control approach. The results of this paper shed light on improving the gliding performance of the robotic dolphin in future mechanical design and gliding control. [ABSTRACT FROM AUTHOR]

In this paper, a nondominated sorting genetic algorithm (NSGA)-II-based calibration algorithm for the underwater binocular vision measurement system is proposed. The underwater measurement results will be incorrect with the binocular camera model in air. To deal with this problem, a refractive camera model and an akin triangulation are proposed to establish the nonlinear relationship with housing parameters between the object and its corresponding image plain points. A novel usage of checkerboard based on the relative position relationship of corners is employed to set three optimal goals, i.e., the distance difference, the vertical direction difference, and the parallel direction difference. The process of calibration is regarded as a multiobjective optimization and solved by NSGA-II. Finally, experimental results demonstrate the validity and effectiveness of the proposed calibration algorithm. [ABSTRACT FROM AUTHOR]