Search

Flying insects employ elegant optical-flow-based strategies to solve complex tasks such as landing or obstacle avoidance. Roboticists have mimicked these strategies on flying robots with only limited success, because optical flow (1) cannot disentangle distance from velocity and (2) is less informative in the highly important flight direction. Here, we propose a solution to these fundamental shortcomings by having robots learn to estimate distances to objects by their visual appearance. The learning process obtains supervised targets from a stability-based distance estimation approach. We have successfully implemented the process on a small flying robot. For the task of landing, it results in faster, smooth landings. For the task of obstacle avoidance, it results in higher success rates at higher flight speeds. Our results yield improved robotic visual navigation capabilities and lead to a novel hypothesis on insect intelligence: behaviours that were described as optical-flow-based and hardwired actually benefit from learning processes. Autonomous flight is challenging for small flying robots, given the limited space for sensors and on-board processing capabilities, but a promising approach is to mimic optical-flow-based strategies of flying insects. A new development improves this technique, enabling smoother landings and better obstacle avoidance, by giving robots the ability to learn to estimate distances to objects by their visual appearance.

International audience; Hybrid unmanned aircraft can significantly increase the potential of micro air vehicles, because they combine hovering capability with a wing for fast and efficient forward flight. However, these vehicles are very difficult to control, because their aerodynamics are hard to model and they are susceptible to wind gusts. This often leads to composite and complex controllers, with different modes for hover, transition, and forward flight. This paper proposes incremental nonlinear dynamic inversion control for the attitude and position control. The result is a single, continuous controller that is able to track the desired acceleration of the vehicle across the flight envelope. The proposed controller is implemented on the Cyclone hybrid unmanned air vehicle. Multiple outdoor experiments are performed, showing that unmodeled forces and moments are effectively compensated by the incremental control structure. Finally, this paper provides a comprehensive procedure for the implementation of the controller on other types of hybrid unmanned air vehicles.

A control strategy is proposed to deal with the fundamental gain selection problem of optical flow landings. It involves detecting the height by means of an oscillating movement and setting the control gains accordingly at the start of a landing. Then, during descent, the gains are reduced exponentially, with mechanisms in place to ensure high-performance landings. Real-world experiments with a quadrotor demonstrate successful landings in both indoor and outdoor environments.

Micro Aerial Vehicles (MAVs) are limited in their operation outdoors near obstacles by their ability to withstand wind gusts. Currently widespread position control methods such as Proportional Integral Derivative control do not perform well under the influence of gusts. Incremental Nonlinear Dynamic Inversion (INDI) is a sensor-based control technique that can control nonlinear systems subject to disturbances. It was developed for the attitude control of manned aircraft or MAVs. In this paper we generalize this method to the outer loop control of MAVs under severe gust loads. Significant improvements over a traditional Proportional Integral Derivative (PID) controller are demonstrated in an experiment where the quadrotor flies in and out of a windtunnel exhaust at 10 m/s. The control method does not rely on frequent position updates, as is demonstrated in an outside experiment using a standard GPS module. Finally, we investigate the effect of using a linearization to calculate thrust vector increments, compared to a nonlinear calculation. The method requires little modeling and is computationally efficient., Comment: 12 pages, 15 figures update: The transfer function in Eq. 12 was incorrect, this has been adjusted. All the conclusions are still valid

Monocular optical flow has been widely used to detect obstacles in Micro Air Vehicles (MAVs) during visual navigation. However, this approach requires significant movement, which reduces the efficiency of navigation and may even introduce risks in narrow spaces. In this paper, we introduce a novel setup of self-supervised learning (SSL), in which optical flow cues serve as a scaffold to learn the visual appearance of obstacles in the environment. We apply it to a landing task, in which initially 'surface roughness' is estimated from the optical flow field in order to detect obstacles. Subsequently, a linear regression function is learned that maps appearance features represented by texton distributions to the roughness estimate. After learning, the MAV can detect obstacles by just analyzing a still image. This allows the MAV to search for a landing spot without moving. We first demonstrate this principle to work with offline tests involving images captured from an on-board camera, and then demonstrate the principle in flight. Although surface roughness is a property of the entire flow field in the global image, the appearance learning even allows for the pixel-wise segmentation of obstacles., This manuscript is submitted to Robotics and Autonomous Systems

In the AlphaPilot Challenge, teams compete to fly autonomous drones through an obstacle course as fast as possible. The 2019 winning team MAVLab reflects on the challenge of beating human pilots.

Swarms of tiny flying robots hold great potential for exploring unknown, indoor environments. Their small size allows them to move in narrow spaces, and their light weight makes them safe for operating around humans. Until now, this task has been out of reach due to the lack of adequate navigation strategies. The absence of external infrastructure implies that any positioning attempts must be performed by the robots themselves. State-of-the-art solutions, such as simultaneous localization and mapping, are still too resource demanding. This article presents the swarm gradient bug algorithm (SGBA), a minimal navigation solution that allows a swarm of tiny flying robots to autonomously explore an unknown environment and subsequently come back to the departure point. SGBA maximizes coverage by having robots travel in different directions away from the departure point. The robots navigate the environment and deal with static obstacles on the fly by means of visual odometry and wall-following behaviors. Moreover, they communicate with each other to avoid collisions and maximize search efficiency. To come back to the departure point, the robots perform a gradient search toward a home beacon. We studied the collective aspects of SGBA, demonstrating that it allows a group of 33-g commercial off-the-shelf quadrotors to successfully explore a real-world environment. The application potential is illustrated by a proof-of-concept search-and-rescue mission in which the robots captured images to find "victims" in an office environment. The developed algorithms generalize to other robot types and lay the basis for tackling other similarly complex missions with robot swarms in the future.

Recently, several insect- and hummingbird-inspired tailless flapping wing robots have been introduced. However, their flight dynamics, which are likely to be similar to that of their biological counterparts, remain yet to be fully understood. We propose a minimal dynamic model that is not only validated with experimental data, but also able to predict the consequences of various important design changes. Specifically, the model captures the flapping-cycle-averaged longitudinal dynamics, considering the main aerodynamic effects. We validated the model with flight data captured with a tailless flapping wing robot, the DelFly Nimble, for air speeds from near-hover flight up to 3.5 m s-1. Moreover, the model succeeds in predicting the effects of changes to the center of mass location, and to the control system gains. Hence, the model is suitable even for the initial control design phase. To demonstrate this, we have used the simulation model to tune the robot's control system for higher speeds. Using the new control parameters on the real robot improved its maximal stable speed from 4 m s-1 to 7 m s-1.

This work proposes PageRank as a tool to evaluate and optimize the global performance of a swarm based on the analysis of the local behavior of a single robot. PageRank is a graph centrality measure that assesses the importance of nodes based on how likely they are to be reached when traversing a graph. We relate this, using a microscopic model, to a random robot in a swarm that transitions through local states by executing local actions. The PageRank centrality then becomes a measure of how likely it is, given a local policy, for a robot in the swarm to visit each local state. This is used to optimize a stochastic policy such that the robot is most likely to reach the local states that are “desirable,” based on the swarm’s global goal. The optimization is performed by an evolutionary algorithm, whereby the fitness function maximizes the PageRank score of these local states. The calculation of the PageRank score only scales with the size of the local state space and demands much less computation than swarm simulations would. The approach is applied to a consensus task, a pattern formation task, and an aggregation task. For each task, when all robots in the swarm execute the evolved policy, the swarm significantly outperforms a swarm that uses the baseline policy. When compared to globally optimized policies, the final performance achieved by the swarm is also shown to be comparable. As this new approach is based on a local model, it natively produces controllers that are flexible and robust to global parameters such as the number of robots in the swarm, the environment, and the initial conditions. Furthermore, as the wall-clock time to evaluate the fitness function does not scale with the size of the swarm, it is possible to optimize for larger swarms at no additional computational expense.

To investigate how an unmanned air vehicle can detect manned aircraft with a single microphone, an audio data set is created in which unmanned air vehicle ego-sound and recorded aircraft sound are mixed together. A convolutional neural network is used to perform air traffic detection. Due to restrictions on flying unmanned air vehicles close to aircraft, the data set has to be artificially produced, so the unmanned air vehicle sound is captured separately from the aircraft sound. They are then mixed with unmanned air vehicle recordings, during which labels are given indicating whether the mixed recording contains aircraft audio or not. The model is a convolutional neural network that uses the features Mel frequency cepstral coefficient, spectrogram or Mel spectrogram as input. For each feature, the effect of unmanned air vehicle/aircraft amplitude ratio, the type of labeling, the window length and the addition of third party aircraft sound database recordings are explored. The results show that the best performance is achieved using the Mel spectrogram feature. The performance increases when the unmanned air vehicle/aircraft amplitude ratio is decreased, when the time window is increased or when the data set is extended with aircraft audio recordings from a third party sound database. Although the currently presented approach has a number of false positives and false negatives that is still too high for real-world application, this study indicates multiple paths forward that can lead to an interesting performance. Finally, the data set is provided as open access.

A time-varying model for the forward flight dynamics of a flapping-wing micro aerial vehicle is identified from free-flight optical tracking data. The model is validated and used to assess the validity of the widely applied time-scale separation assumption. Based on this assumption, each aerodynamic force and moment is formulated as a linear addition of decoupled time-averaged and time-varying submodels. The resulting aerodynamic models are incorporated in a set of linearized equations of motion, yielding a simulation-capable full dynamic model. The time-averaged component includes both the longitudinal and the lateral aerodynamics and is assumed to be linear. The time-varying component is modeled as a third-order Fourier series, which approximates the flapping dynamics effectively. Combining both components yields a more complete and realistic simulation. Results suggest that while in steady flight the time-scale separation assumption applies well during maneuvers the time-varying dynamics are not fully captured. More accurate modeling of flapping-wing flight during maneuvers may require considering coupling between the time scales.

Autonomous flight of Flapping Wing Micro Air Vehicles (FWMAVs) is a major challenge in the field of robotics, due to their light weight and their flapping-induced body motions. An FWMAV is presented weighing a mere 20 g while all its sensors and processing for autonomous flight are onboard. The navigation is based on a 4-g stereo vision camera with onboard processing. Three basic navigational tasks are demonstrated, namely obstacle avoidance, door traversing and corridor following. The presented combination of sensors and control routines is shown to allow flight in common unprepared environments like corridors and offices. The algorithms do not depend on prior classification or learning of the environment or control logic and work in any unprepared environment with vertical texture. While some failure cases remain, this work forms an important step towards very small autonomous indoor MAV.

This paper presents a literature survey and a comparative study of Bug Algorithms, with the goal of investigating their potential for robotic navigation. At first sight, these methods seem to provide an efficient navigation paradigm, ideal for implementations on tiny robots with limited resources. Closer inspection, however, shows that many of these Bug Algorithms assume perfect global position estimate of the robot which in GPS-denied environments implies considerable expenses of computation and memory -- relying on accurate Simultaneous Localization And Mapping (SLAM) or Visual Odometry (VO) methods. We compare a selection of Bug Algorithms in a simulated robot and environment where they endure different types noise and failure-cases of their on-board sensors. From the simulation results, we conclude that the implemented Bug Algorithms' performances are sensitive to many types of sensor-noise, which was most noticeable for odometry-drift. This raises the question if Bug Algorithms are suitable for real-world, on-board, robotic navigation as is. Variations that use multiple sensors to keep track of their progress towards the goal, were more adept in completing their task in the presence of sensor-failures. This shows that Bug Algorithms must spread their risk, by relying on the readings of multiple sensors, to be suitable for real-world deployment., Comment: 22 pages, 20 figures, Submitted to Robotics and Autonomous Systems

This paper presents an approach to the system identification of the Delfly II Flapping Wing Micro Air Vehicle (FWMAV) using flight test data. It aims at providing simple FWMAV aerodynamic models that can be used in simulations as well as in nonlinear flight control systems. The undertaken methodology builds on normal aircraft system identification methods and extends these with techniques that are specific to FWMAV model identification. The entire aircraft model identification cycle is discussed covering the set-up and automatic execution of the flight test experiments, the aircraft states, the aerodynamic forces and moments' reconstruction, the aerodynamic model structure selection, the parameter estimation and finally, the model validation. In particular, a motion capturing facility was used to record the flapper's position in time and from there compute the states and aerodynamic forces and moments that acted on it, assuming flap-averaged dynamics and linear aerodynamic model structures. It is shown that the approach leads to aerodynamic models that can predict the aerodynamic forces with high accuracy. Despite less accurate, the predictions of the aerodynamic moments still follow the general trend of the measured moments. Dynamic simulations based on the identified aerodynamic models show flight trajectories that closely match the ground truth spanning a number of flapping cycles. Finally, the dimensional aerodynamic forces and moments' coefficients of two of the identified aerodynamic models are presented.

An Evolutionary Robotics (ER) approach to the task of odor source localization is investigated. In particular, Continuous Time Recurrent Neural Networks (CTRNNs) are evolved for odor source localization in simulated turbulent odor plumes. In the experiments, the simulated robot is equipped with a single chemical sensor and a wind direction sensor. Three main contributions are made. First, it is shown that the ER approach can be successfully applied to odor source localization in both low-turbulent and high-turbulent conditions. Second, it is demonstrated that a small neural network is able to successfully perform all three sub-tasks of odor source localization: (i) finding the odor plume, (ii) moving toward the odor source, and (iii) identifying the odor source. Third, the analysis of the evolved behaviors reveals two novel odor source localization strategies. These strategies are successfully re-implemented as finite state machines, validating the insights from the analysis of the neural controllers.

This work extends the drag-force enhanced quadrotor model by denoting the free stream air velocity as the difference between the ground speed and the wind speed. It is demonstrated that a relatively simple nonlinear observer is capable of estimating the local wind components, provided accelerometer and GPS-velocity measurements are available. We perform a wind tunnel experiment at various wind speeds using a quadrotor vehicle with a low-cost Inertial Measurement Unit (IMU) and a motion tracking system to provide accurate ground speed measurements. It is shown that the onboard Extended Kalman Filter (EKF) accurately estimates the wind components.

The study investigates the aerodynamic performance and the relation between wing deformation and unsteady force generation of a flapping-wing micro air vehicle in hovering flight configuration. Different experiments were performed where fluid forces were acquired with a force sensor, while the three-dimensional wing deformation was measured with a stereo-vision system. In these measurements, time-resolved power consumption and flapping-wing kinematics were also obtained under both in-air and in-vacuum conditions. Comparison of the results for different flapping frequencies reveals different wing kinematics and deformation characteristics. The high flapping frequency case produces higher forces throughout the complete flapping cycle. Moreover, a phase difference occurs in the variation of the forces, such that the low flapping frequency case precedes the high frequency case. A similar phase lag is observed in the temporal evolution of the wing deformation characteristics, suggesting that there is a direct link between the two phenomena. A considerable camber formation occurs during stroke reversals, which is mainly determined by the stiffener orientation. The wing with the thinner surface membrane displays very similar characteristics to the baseline wing, which implies the dominance of the stiffeners in terms of providing rigidity to the wing. Wing span has a significant effect on the aerodynamic efficiency such that increasing the span length by 4 cm results in a 6% enhancement in the cycle-averaged X-force to power consumption ratio compared to the standard DelFly II wings with a span length of 28 cm.

Flapping-wing aerodynamic models that are accurate, computationally efficient and physically meaningful, are challenging to obtain. Such models are essential to design flapping-wing micro air vehicles and to develop advanced controllers enhancing the autonomy of such vehicles. In this work, a phenomenological model is developed for the time-resolved aerodynamic forces on clap-and-fling ornithopters. The model is based on quasi-steady theory and accounts for inertial, circulatory, added mass and viscous forces. It extends existing quasi-steady approaches by: including a fling circulation factor to account for unsteady wing-wing interaction, considering real platform-specific wing kinematics and different flight regimes. The model parameters are estimated from wind tunnel measurements conducted on a real test platform. Comparison to wind tunnel data shows that the model predicts the lift forces on the test platform accurately, and accounts for wing-wing interaction effectively. Additionally, validation tests with real free-flight data show that lift forces can be predicted with considerable accuracy in different flight regimes. The complete parameter-varying model represents a wide range of flight conditions, is computationally simple, physically meaningful and requires few measurements. It is therefore potentially useful for both control design and preliminary conceptual studies for developing new platforms.

Monocular vision is increasingly used in micro air vehicles for navigation. In particular, optical flow, inspired by flying insects, is used to perceive vehicle movement with respect to the surroundings or sense changes in the environment. However, optical flow does not directly provide us the distance to an object or velocity, but the ratio of them. Thus, using optical flow in control involves nonlinearity problems which add complexity to the controller. To deal with that, we propose an algorithm that estimates distance and velocity of the vehicle based on optical flow measured from a monocular camera and the knowledge of control inputs. This algorithm applies an extended Kalman filter to state estimation and uses the estimates for landing control. We implement and test our algorithm in computer simulation and on board a Parrot AR.Drone 2.0 to demonstrate its feasibility for micro air vehicles landings. Results of the simulation and multiple flight tests show that the algorithm is able to estimate height and velocity of the micro air vehicles accurately, and achieves smooth landings with these estimates, even in windy outdoor environments.

Unmanned Aerial Vehicles (UAVs) can be used to access remote areas that were otherwise inaccessible, for example, for surveillance missions. Collaboration between them can help overcome communication constraints by building airborne relay networks that allow beyond line of sight communication. This research investigates if a single operator can supervise multiple UAVs in a collaborative surveillance task under communication constraints. For this purpose and ecological interface was designed to support operators in their task and to bring flexibility in the system. A human-in-the-loop evaluation study was performed to investigate the successfulness of operators in the control task of such a mission including an analysis of individual components of the interface. It was shown that operators are able to successfully operate surveillance missions under communication- and battery constraints. Participants did however not completely do this without separation conflicts and communication losses, which indicates that the interface lacks elements representing endurance and separation assurance. To an extent the interface design turned out to be scalable, with a few remaining visualizations that still suffered from this problem. More advanced ways of displaying information on request and grouping of select information is thought to offer opportunities to improve ground control interface on this matter.

The appearance variation cue captures the variation in texture in a single image. Its use for obstacle avoidance is based on the assumption that there is less such variation when the camera is close to an obstacle. For videos of approaching frontal obstacles, it is demonstrated that combining the cue with optic flow leads to better performance than using either cue alone. In addition, the cue is successfully used to control the 16-g flapping-wing micro air vehicle DelFly II.

Small robotic systems such as Micro Air Vehicles (MAVs) need to react quickly to their dynamic environments, while having only a limited amount of energy and processing onboard. In this article, sub-sampling of local image samples is investigated as a straightforward and broadly applicable approach to improve the computational efficiency of vision algorithms. In sub-sampling, only a small subset of the total number of samples is processed, leading to a significant reduction of the computational effort at the cost of a slightly lower accuracy. The possibility to change the number of extracted samples is of particular importance to autonomous robots, since it allows the designer to select not only the performance but also the execution frequency of the algorithm. The approach of sub-sampling is illustrated by introducing two novel, computationally efficient algorithms for two tasks relevant to MAVs: WiFi noise detection in camera images and onboard horizon detection for pitch and roll estimation. In the noise detection task, image lines and pixel pairs are sampled, while in the horizon detection task features from local image patches are sampled. For both tasks experiments are performed and the effects of sub-sampling are analyzed. It is demonstrated that even for small images of size 160x120 speed-ups of a factor 14 to 21 are reached, while retaining a sufficient performance for the tasks at hand.

Light-weight, autonomous ornithopters form a promise to observe places that are too small or too dangerous for humans to enter. In this article, we discuss the DelFly project, in which we follow a top-down approach to ever smaller and more autonomous ornithopters. Top-down signifies that the project always focuses on complete flying systems equipped with camera. We give arguments for the approach by explaining which findings on the DelFly I and DelFly II recently led to the development of the DelFly Micro: a 3.07-gram ornithopter carrying a camera and transmitter onboard. These findings concern the design, aerodynamics, and vision-based control of the DelFly. In addition, we identify main obstacles on the road to fly-sized ornithopters.

We shift our focus to the use of stereo vision for autonomous flight. Stereo vision implies carrying two cameras on board, which adds weight and increases the power consumption. Still, it also allows for instantaneous distance estimates, which is a considerable advantage on a moving (and oscillating) flapping wing MAV. In particular, we explain the onboard stereo vision and control algorithms that allow the 20-g DelFly Explorer to autonomously fly around in unknown environments for as long as its battery lasts.

In this chapter, we first explain the choice for the X-wing configuration of the DelFly. Subsequently, we discuss the evolution of the different mechanical parts used in the DelFly I, II, Micro, and Explorer. Details are given on the used materials.

In this chapter, an overview of the experimental studies focusing on the aerodynamics of the DelFly is given. Force generation mechanisms of the DelFly flapping-wing flight are addressed with particular consideration of the effects of different parameters such as wing geometry and flexibility. Furthermore, state-of-art flow visualization results are presented to give insight into the behavior of flow structures around and in the wake of the flapping wings and to assess their influence on the unsteady forces.

This chapter treats the main choices, issues, and tradeoffs in the design of flapping wing MAVs. In particular, we discuss the implications of different tail and wing configurations, the energy source and various types of actuators. We also show how choices elementary to aircraft design, such as the trade-off between fuel/battery mass and payload mass can have rather large effects at the scale of light-weight flapping wing MAVs.

The DelFly does not use a map of its environment to achieve successful obstacle avoidance. Instead, it uses more direct visual cues to detect obstacles and to turn in the right direction. In this chapter, we investigate how successful such a direct avoidance strategy can be. We introduce a turning logic for the DelFly and test it both in simulation and on the real DelFly II platform, confirming the validity of the approach and identifying some of the challenges still ahead.

In this chapter, physical principles of fixed-wing and flapping-wing aerodynamics are introduced briefly with the associated terminology. Furthermore, the main aerodynamic mechanisms which are responsible for the generation of the forces in the flapping-wing flight are described in order to provide the reader with fundamental knowledge before further comprehensive analysis of the DelFly.

This chapter sets the stage for the research on autonomous flight of the DelFly. First, a general introduction is given to artificial intelligence for robotics. This will permit the layman to understand some of the major challenges in creating autonomous robots, and the different solution approaches. Subsequently, the particular tasks and approaches to autonomous flight of Micro Air Vehicles are discussed. Whereas mainstream approaches are now being applied to \(\sim \)1 kg MAVs such as quadrotors, it will become clear that light-weight flapping wing MAVs are best served by a computationally efficient, vision-based approach to autonomous flight. In particular, in the DelFly project we have adopted a purposive vision approach to autonomous flight, in which only the information necessary for the robot’s task is extracted. The main goal of the approach is to allow the DelFly to autonomously explore unknown indoor environments, to which end we complement optical flow with other, appearance based, visual inputs.

This chapter deals with monocular obstacle detection. The visual cue of optic flow is used to determine the time-to-impact to obstacles in the environment. Since the flapping wing motion hampers the determination of optic flow, a complementary, “appearance variation cue” is studied. Combining these visual cues significantly improves detection results.

We introduce a novel setup of self-supervised learning (SSL), in which optical flow provides the supervised outputs. Optical flow requires significant movement for obstacle detection. The main advantage of the introduced method is that after learning, a robot can detect obstacles without moving - reducing the risk of collisions in narrow spaces. We investigate this novel setup of SSL in the context of a Micro Air Vehicle (MAV) that needs to select a suitable landing place. Initially, when the MAV flies over a potential landing area, the optical flow processing estimates a ‘surface roughness’ measure, capturing whether there are obstacles sticking out of the landing surface. This measure allows the MAV to select a safe landing place and then land with other optical flow measures such as the divergence. During flight, SSL takes place. For each image a texton distribution is extracted (capturing the visual appearance of the landing surface in sight), and mapped to the current roughness value by a linear regression function. We first demonstrate this principle to work with offline tests involving images captured on board an MAV, and then demonstrate the principle in flight. The experiments show that the MAV can land safely on the basis of optical flow. After learning it can also successfully select safe landing spots in hover. It is even shown that the appearance learning allows the pixel-wise segmentation of obstacles.

The autonomous capabilities of light-weight Flapping Wing Micro Air Vehicles (FWMAVs) have much to gain from onboard state estimation and attitude control. In this article, we present the first FWMAV with robust onboard state estimation and attitude control. The tailed FWMAV DelFly II was used, with the main goal to achieve active stabilization in the (passively unstable) hover condition. The attitude is estimated using an Inertial Measurement Unit with a gyroscope, accelerometer and magnetometer and the altitude is estimated using a barometer. A major challenge lies in the disturbance of the accelerometer measurements by the flapping motion of the wings. We propose a mechanical damping mechanism and flap-cycle based filtering to resolve this issue. The pitch estimates have a mean error of 1.5° with respect to the ground-truth measurement from a motion capture system. Using the onboard pitch estimate we can control the attitude of the FWMAV in the forward flight regime with a 30% lower standard deviation than in a trimmed flight. With a different set of gains, the FWMAV is able to perform a hovering flight - showing that a tailed FWMAV has enough control authority for this task. In a fully autonomous hover experiment, the DelFly II stays within a sphere of 0.75 m radius.

The determination of dynamic forces acting on a Flapping Wing Micro Aerial Vehicle (FWMAV) is a challenging task due to the unsteady nature of force generation mechanisms. To assure a proper force identification in future researches, this work compares two different methods to obtain the longitudinal forces acting on FWMAVs and discusses their applicability regions. The methods were 1) calculation of forces from the recordings of the FWMAV's position in a free flight condition; 2) direct force measurements in a tethered flight condition in a wind tunnel. The DelFly II is used as the FWMAV test platform in the measurements. During free flight experiments, its position and attitude were recorded at a rate of 200Hz using an external visual tracking system, whose acquired information was then analyzed to obtain the flight states and calculate the forces and moments that act on the platform during flight, under a set of kinematic assumptions. Subsequently, similar flight conditions were tested in the tethered situation. An ATI Nano-17 Titanium force transducer was used to measure time-resolved forces. The results for the most common flight regime of the DelFly, which is a slow forward flight at a high body pitch angle, are presented. It is shown that the tethered force balance tests agree with the free flight data when assessing the aerodynamic forces that are perpendicular to the stroke plane of the flapping wing. However, the forces that act along the stroke plane are coupled with structural dynamic terms, thus affecting the final lift and thrust identification. These results point to inadequate force identification in fixed point force measurements, due to effect the of the dynamic modes of the FWMAV body, thus advising proper cross-comparing between experimental methods.

Autonomous flight of Flapping Wing Micro Air Vehicles (FWMAVs) is a major challenge in the field of robotics, due to their light weight and the flapping-induced body motions. In this article, we present the first FWMAV with onboard vision processing for autonomous flight in generic environments. In particular, we introduce the DelFly ‘Explorer’, a 20-gram FWMAV equipped with a 0.98-gram autopilot and a 4.0-gram onboard stereo vision system. We explain the design choices that permit carrying the extended payload, while retaining the DelFly’s hover capabilities. In addition, we introduce a novel stereo vision algorithm, LongSeq, designed specifically to cope with the flapping motion and the desire to attain a computational effort tuned to the frame rate. The onboard stereo vision system is illustrated in the context of an obstacle avoidance task in an environment with sparse obstacles.

Several studies have focused on deriving kinematic formulations of simulated flapping wing micro air vehicles (FWMAV). However, very few used real-life flight data and none have compared the predicted aerodynamic forces and moments across different kinematic formulation principles. Hence, the present study compares and assesses the quality of simple equations of motion against complex multi-body formulations using real flight data. In particular, the position and attitude of an autonomous flying FWMAV was logged by an external high resolution visual tracking system. States were reconstructed using flight path reconstruction techniques and were used as inputs for the determination of the aerodynamic forces and moments that acted on the FWMAV. Two kinematic models of a 4 wing FWMAV were derived and used to compute the aerodynamic forces and moments: 1) simple Newton-Euler formulation of rigid aircraft equations of motion; 2) complex 5 body kinematic model using D'Alembert's principle. The results are presented for trimmed flight, as well as for system identification maneuvers characterized by doublet inputs on the rudder and elevator. These results show the difference between both formulations and indicate that rigid body kinematic forms can be used for FWMAV aerodynamic system identification, with the advantageous use in control design of iterative versions of ornithopters, showing an average correlation of 0.98 (out of 1) with more complex formulations. Multi-body kinematics, on the other hand, despite more time-expensive to derive, capture more contributions of the different FWMAV structures as well as the internal forces and moments (like driving motor torque), thus being more suitable for simulation and robust control of complex ornithopters.

Micro Air Vehicles need to have a robust landing capability, especially when they operate outside line-of-sight. Autonomous landing requires the identification of a relatively flat landing surface that does not have too large an inclination. In this article, a vision algorithm is introduced that fits a second-order approximation to the optic flow field underlying the optic flow vectors in images from a bottom camera. The flow field provides information on the ventral flow ( Vx/h), the time-to-contact ( h/ – Vz), the flatness of the landing surface, and the surface slope. The algorithm is computationally efficient and since it regards the flow field as a whole, it is suitable for use during relatively fast maneuvers. The algorithm is subsequently tested on artificial image sequences, hand-held videos, and on the images made by a Parrot AR drone. In a preliminary robotic experiment, the AR drone uses the vision algorithm to determine when to land in a scenario where it flies off a stairs onto the flat floor.

One of the major challenges in robotics is to develop a fly-like robot that can autonomously fly around in unknown environments. In this paper, we discuss the current state of the DelFly project, in which we follow a top-down approach to ever smaller and more autonomous ornithopters. The presented findings concerning the design, aerodynamics and autonomy of the DelFly illustrate some of the properties of the top-down approach, which allows the identification and resolution of issues that also play a role at smaller scales. A parametric variation of the wing stiffener layout produced a 5% more power-efficient wing. An experimental aerodynamic investigation revealed that this could be associated with an improved stiffness of the wing, while further providing evidence of the vortex development during the flap cycle. The presented experiments resulted in an improvement in the generated lift, allowing the inclusion of a yaw rate gyro, pressure sensor and microcontroller onboard the DelFly. The autonomy of the DelFly is expanded by achieving (1) an improved turning logic to obtain better vision-based obstacle avoidance performance in environments with varying texture and (2) successful onboard height control based on the pressure sensor.

The capability to visually discern possible obstacles from the sky would be a valuable asset to a UAV for avoiding both other flying vehicles and static obstacles in its environment. The main contribution of this article is the presentation of a feasible approach to obstacle avoidance based on the segmentation of camera images into sky and non-sky regions. The approach is named the Sky Segmentation Approach (SSA). The central concept is that potentially threatening static obstacles protrude from the horizon line. The main challenge for SSA is automatically interpreting the images robustly enough for use in various environments and fast enough for real-time performance. In order to achieve robust image segmentation, machine learning is applied to a large database of images with many different types of skies. From these images, different types of visual features are extracted, among which most of the features investigated in the literature. In the interest of execution speed and comprehensibility, decision trees are learned to map the feature values at an image location to a classification as sky or non-sky. The learned decision trees are fast enough to allow real-time execution on a Digital Signal Processor: it is run onboard a small UAV at ∼ 30 Hz. Experiments in simulation and preliminary experiments on a small UAV show the potential of SSA for achieving robust obstacle avoidance in urban areas.

An accurate knowledge of the unsteady aerodynamic forces acting on a bio-inspired, flapping-wing micro air vehicle (FWMAV) is crucial in the design development and optimization cycle. Two different types of experimental approaches are often used: determination of forces from position data obtained from external optical tracking during free flight, or direct measurements of forces by attaching the FWMAV to a force transducer in a wind-tunnel. This study compares the quality of the forces obtained from both methods as applied to a 17.4 gram FWMAV capable of controlled flight. A comprehensive analysis of various error sources is performed. The effects of different factors, e.g., measurement errors, error propagation, numerical differentiation, filtering frequency selection, and structural eigenmode interference, are assessed. For the forces obtained from free flight experiments it is shown that a data acquisition frequency below 200 Hz and an accuracy in the position measurements lower than ± 0.2 mm may considerably hinder determination of the unsteady forces. In general, the force component parallel to the fuselage determined by the two methods compares well for identical flight conditions; however, a significant difference was observed for the forces along the stroke plane of the wings. This was found to originate from the restrictions applied by the clamp to the dynamic oscillations observed in free flight and from the structural resonance of the clamped FWMAV structure, which generates loads that cannot be distinguished from the external forces. Furthermore, the clamping position was found to have a pronounced influence on the eigenmodes of the structure, and this effect should be taken into account for accurate force measurements.

A visual cue is introduced that exploits the visual appearance of a single image to estimate the proximity to an obstacle. In particular, the appearance variation cue captures the variation in texture and / or color in the image, and is based on the assumption that there is less such variation when the camera is close to an obstacle. Random sampling is applied in order to evaluate the appearance variation fast enough for use in robotics. It is demonstrated that the randomly sampled appearance variation cue can be complementary to optic flow for obstacle detection; combining the two visual cues leads to better obstacle detection performance. Random sampling leads to sufficient computational efficiency for the cue's utilization in autonomous flight: a speed-up of a factor ∼100 is attained, which allows the successful control of the 16-gram flapping wing MAV DelFly II.

Landing is arguably one of the most important behaviors that flying animals regularly perform. It involves a precise control of approach speed as an animal draws closer to the landing surface. Poor control can result in high-impact collisions with the surface which can be harmful for animals. Despite its importance in flight, how animals approach a surface for landing is not yet fully understood. In this thesis, I contribute to answering this question by examining the landing approaches of bumblebees and honeybees. Bees, including bumblebees and honeybees, perform 100 to 1000 landings in a single hour of foraging. They perform these landings relentlessly to gather nectar and pollen from flowers which are essential for their survival and reproduction. In this thesis, I use novel analyses methods to investigate how these bees land.Many flying animals use visual cues during landing. In chapter 2, we present how bumblebees use visual expansion cues to advance towards the landing surface. For this purpose, we first designed an indoor experimental apparatus to automatically record the landing maneuvers of foraging bumblebees. We then analyzed 4,672 landing maneuvers using a novel method. This method analyses the individual maneuvers and is more comprehensive than the analysis method of averaging multiple maneuvers used in literature. Using this novel method, we found the visual guidance strategy of landing bumblebees. Our results show that the landing bumblebees exhibit a series of deceleration bouts during which they keep the relative rate of optical expansion approximately constant. This constant is referred to as a set-point and from one bout to the next, bumblebees tend to shift to a higher set-point. This newly-found guidance strategy results in the approach dynamics that is strikingly similar to that of pigeons and hummingbirds. In addition, we also found how bumblebees adjust this visual guidance strategy to travel faster when landing directly after a take-off than from a free-flight condition. Moreover, we also elucidated how bumblebees adjust this guidance strategy in the presence of different strength of optic expansion cues available from the landing surface (checkerboard versus spoke patterns) and different light intensities ranging from twilight to sunrise. This guidance strategy helps to explain how these important pollinators rapidly visit flowers and forage in challenging environmental conditions.In addition to the deceleration phases, we found that landing bumblebees also occasionally exhibit low approach velocity phases (V < 0.05 m s-1) while transitioning from one set-point to another. These low approach velocity phases are similar to the hovering phases identified in literature, and result in bumblebees hovering or sometimes even flying away from the surface for a short while. In chapter 2, we also proposed that these low approach velocity phases are likely the instabilities arising out of a control system that uses optical expansion rate as a control variable.For achieving a goal such as evading a threat or reaching a set-point, animals use their sensorimotor control system to continuously parse the sensory information and change the wingbeat and body kinematics to produce the required forces and torques. In chapter 3, we focused on the sensorimotor control system that landing bumblebees use to execute their visual guidance strategy. We used the natural stepwise excitation that landing bumblebees offer to analyze how their different subsystems (sensory system, controller and motor system) function together to reach the set-points of optical expansion rate. Our results showed that their closed-loop sensorimotor control system regulates the relative rate of expansion during landing. The track segments before and during a set-point are the transient and steady-state responses of such a control system. Bumblebees use the transient response to mostly accelerate and steady-state response to always decelerate during their landing approach. We also identified how the transient response varies among the tested environmental conditions (light intensity and the strength of optic expansion cues) and starting conditions (landings from a free-flight or after a take-off). Based on these results, we propose a sensorimotor control system of landing bumblebees that facilitates a rapid and robust execution of their visual guidance strategy.Bumblebees regularly experience winds during foraging. The winds in nature can be characterized as mean winds and fluctuations around them. In chapter 4, we investigated how the mean winds affect the visual guidance strategy, the sensorimotor control system and the landing performance of foraging bumblebees. In particular, we used six steady sidewinds ranging from 0 – 3.41m s-1 that foraging bumblebees often encounter. We found that the visual guidance strategy and the sensorimotor control response of bumblebees in these wind conditions is similar to the still-air response, but bumblebees exhibit some important adaptations in winds. Compared to the still-air situation, bumblebees more often exhibit low approach velocity phases in higher wind speeds. This can lead to an increase in the travel time and hence, can adversely affect their foraging efficiency. But, bumblebees also exhibit faster transient responses and higher set-points with increasing wind speed which enable them to travel faster. This in turn allows bumblebees to compensate for the increase in travel time that would otherwise occur due to more low approach-velocity phases in higher winds. In addition to revealing the adverse effects of winds and the compensation mechanism of bumblebees during landing, we also use the natural excitation of the sensorimotor control system that bumblebees offer during landing to propose how they integrate information from the airspeed measuring mechanosensors with their visual feedback loop.In chapter 5, we revise the visual guidance strategy of landing honeybees previously proposed in literature. In literature, honeybees are shown to linearly decrease their approach velocity with the reducing distance by analyzing the average of multiple landing maneuvers. Based on this result, it has been suggested that they land by holding the relative rate of optical expansion constant throughout their approach. We use the novel analysis technique developed in chapter 2 to show that the individual honeybees do not follow such a strategy. They instead stepwise modulate their set-point of optical expansion rate during landing. Moreover, we extend the analysis to find the mechanism that allows honeybees to converge to a stereotypic landing maneuver closer to the landing surface, for a large range of initial flight speeds and visual landing platform patterns.Finally, in chapter 6, I synthesize the results of this thesis and place them in a broader context of flight control in bees specifically, and other flying animals in general. For this, I first compare the landing strategies of bumblebees and honeybees found in this thesis and elucidate the likely causes of the differences between their strategies. Then, I discuss how birds might perform control during landing. Additionally, I also discuss how the knowledge obtained in this thesis can be used for bioinspired applications. Finally, I present an outlook on the future research in the area of landing dynamics of insects and birds.Considering all results together, in this thesis, we developed and used a novel analysis to demonstrate that bumblebees and honeybees have evolved a sophisticated flight control strategy to execute rapid landings. Moreover, we have shown that they have evolved ways to adjust this modular guidance strategy to deal with the challenges offered by the environment