Search

Recent spacecraft mission concepts propose larger payloads that have lighter, less rigid structures. For large lightweight structures, the natural frequencies of their vibration modes may fall within the attitude controller bandwidth, threatening the stability and settling time of the controller and compromising performance. This work tackles this issue by proposing an attitude control design paradigm of distributing momentum actuators throughout the structure to have more control authority over vibration modes. The issue of jitter disturbances introduced by these actuators is addressed by expanding the bandwidth of the attitude controller to suppress excess vibrations. Numerical simulation results show that, at the expense of more control action, a distributed configuration can achieve lower settling times and reduce structural deformation compared to a more standard centralized configuration., Comment: 10 pages, 9 figures

The integration of Artificial Intelligence (AI) and Augmented Reality (AR) is set to transform satellite Assembly, Integration, and Testing (AIT) processes by enhancing precision, minimizing human error, and improving operational efficiency in cleanroom environments. This paper presents a technical description of the European Space Agency's (ESA) project "AI for AR in Satellite AIT," which combines real-time computer vision and AR systems to assist technicians during satellite assembly. Leveraging Microsoft HoloLens 2 as the AR interface, the system delivers context-aware instructions and real-time feedback, tackling the complexities of object recognition and 6D pose estimation in AIT workflows. All AI models demonstrated over 70% accuracy, with the detection model exceeding 95% accuracy, indicating a high level of performance and reliability. A key contribution of this work lies in the effective use of synthetic data for training AI models in AR applications, addressing the significant challenges of obtaining real-world datasets in highly dynamic satellite environments, as well as the creation of the Segmented Anything Model for Automatic Labelling (SAMAL), which facilitates the automatic annotation of real data, achieving speeds up to 20 times faster than manual human annotation. The findings demonstrate the efficacy of AI-driven AR systems in automating critical satellite assembly tasks, setting a foundation for future innovations in the space industry.

Pseudo-haptic techniques are becoming increasingly popular in human-computer interaction. They replicate haptic sensations by leveraging primarily visual feedback rather than mechanical actuators. These techniques bridge the gap between the real and virtual worlds by exploring the brain's ability to integrate visual and haptic information. One of the many advantages of pseudo-haptic techniques is that they are cost-effective, portable, and flexible. They eliminate the need for direct attachment of haptic devices to the body, which can be heavy and large and require a lot of power and maintenance. Recent research has focused on applying these techniques to extended reality and mid-air interactions. To better understand the potential of pseudo-haptic techniques, the authors developed a novel taxonomy encompassing tactile feedback, kinesthetic feedback, and combined categories in multimodal approaches, ground not covered by previous surveys. This survey highlights multimodal strategies and potential avenues for future studies, particularly regarding integrating these techniques into extended reality and collaborative virtual environments., Comment: 26 pages, 6 figures, accepted for publication in IEEE Access

Time perception - how humans and animals perceive the passage of time - forms the basis for important cognitive skills such as decision-making, planning, and communication. In this work, we propose a framework for examining the mechanisms responsible for time perception. We first model neural time perception as a combination of two known timing sources: internal neuronal mechanisms and external (environmental) stimuli, and design a decision-making framework to replicate them. We then implement this framework in a simulated robot. We measure the agent's success on a temporal discrimination task originally conducted by mice to evaluate its capacity to exploit temporal knowledge. We conclude that the agent is able to perceive time similarly to animals when it comes to their intrinsic mechanisms of interpreting time and performing time-aware actions. Next, by analysing the behaviour of agents equipped with the framework, we propose an estimator to infer characteristics of the timing mechanisms intrinsic to the agents. In particular, we show that from their empirical action probability distribution we are able to estimate parameters used for perceiving time. Overall, our work shows promising results when it comes to drawing conclusions regarding some of the characteristics present in biological timing mechanisms.

On-orbit close proximity operations involve robotic spacecraft maneuvering and making decisions for a growing number of mission scenarios demanding autonomy, including on-orbit assembly, repair, and astronaut assistance. Of these scenarios, on-orbit assembly is an enabling technology that will allow large space structures to be built in-situ, using smaller building block modules. However, robotic on-orbit assembly involves a number of technical hurdles such as changing system models. For instance, grappled modules moved by a free-flying "assembler" robot can cause significant shifts in system inertial properties, which has cascading impacts on motion planning and control portions of the autonomy stack. Further, on-orbit assembly and other scenarios require collision-avoiding motion planning, particularly when operating in a "construction site" scenario of multiple assembler robots and structures. These complicating factors, relevant to many autonomous microgravity robotics use cases, are tackled in the ReSWARM flight experiments as a set of tests on the International Space Station using NASA's Astrobee robots. RElative Satellite sWarming and Robotic Maneuvering, or ReSWARM, demonstrates multiple key technologies for close proximity operations and on-orbit assembly: (1) global long-horizon planning, accomplished using offline and online sampling-based planner options that consider the system dynamics; (2) on-orbit reconfiguration model learning, using the recently-proposed RATTLE information-aware planning framework; and (3) robust control tools to provide low-level control robustness using current system knowledge. These approaches are detailed individually and in an "on-orbit assembly scenario" of multi-waypoint tracking on-orbit. Additionally, detail is provided discussing the practicalities of hardware implementation and unique aspects of working with Astrobee in microgravity.

Certain forms of uncertainty that robotic systems encounter can be explicitly learned within the context of a known model, like parametric model uncertainties such as mass and moments of inertia. Quantifying such parametric uncertainty is important for more accurate prediction of the system behavior, leading to safe and precise task execution. In tandem, providing a form of robustness guarantee against prevailing uncertainty levels like environmental disturbances and current model knowledge is also desirable. To that end, the authors' previously proposed RATTLE algorithm, a framework for online information-aware motion planning, is outlined and extended to enhance its applicability to real robotic systems. RATTLE provides a clear tradeoff between information-seeking motion and traditional goal-achieving motion and features online-updateable models. Additionally, online-updateable low level control robustness guarantees and a new method for automatic adjustment of information content down to a specified estimation precision is proposed. Results of extensive experimentation in microgravity using the Astrobee robots aboard the International Space Station and practical implementation details are presented, demonstrating RATTLE's capabilities for real-time, robust, online-updateable, and model information-seeking motion planning capabilities under parametric uncertainty., Comment: 8 pages, 11 figures, RA-L with IROS 2022 option

With deep learning gaining attention from the research community for prediction and control of real physical systems, learning important representations is becoming now more than ever mandatory. It is of extreme importance that deep learning representations are coherent with physics. When learning from discrete data this can be guaranteed by including some sort of prior into the learning, however, not all discretization priors preserve important structures from the physics. In this paper, we introduce Symplectic Momentum Neural Networks (SyMo) as models from a discrete formulation of mechanics for non-separable mechanical systems. The combination of such formulation leads SyMos to be constrained towards preserving important geometric structures such as momentum and a symplectic form and learn from limited data. Furthermore, it allows to learn dynamics only from the poses as training data. We extend SyMos to include variational integrators within the learning framework by developing an implicit root-find layer which leads to End-to-End Symplectic Momentum Neural Networks (E2E-SyMo). Through experimental results, using the pendulum and cartpole, we show that such combination not only allows these models to learn from limited data but also provides the models with the capability of preserving the symplectic form and show better long-term behaviour., Comment: 12 pages, 4 figures. Accepted at 4th Annual Learning for Dynamics & Control Conference

Space free-flyers like the Astrobee robots currently operating aboard the International Space Station must operate with inherent system uncertainties. Parametric uncertainties like mass and moment of inertia are especially important to quantify in these safety-critical space systems and can change in scenarios such as on-orbit cargo movement, where unknown grappled payloads significantly change the system dynamics. Cautiously learning these uncertainties en route can potentially avoid time- and fuel-consuming pure system identification maneuvers. Recognizing this, this work proposes RATTLE, an online information-aware motion planning algorithm that explicitly weights parametric model-learning coupled with real-time replanning capability that can take advantage of improved system models. The method consists of a two-tiered (global and local) planner, a low-level model predictive controller, and an online parameter estimator that produces estimates of the robot's inertial properties for more informed control and replanning on-the-fly; all levels of the planning and control feature online update-able models. Simulation results of RATTLE for the Astrobee free-flyer grappling an uncertain payload are presented alongside results of a hardware demonstration showcasing the ability to explicitly encourage model parametric learning while achieving otherwise useful motion., Comment: 8 pages, 8 figures, IROS 2021 preprint (accepted)

We introduce a closed-form method for identification of discrete-time linear time-variant systems from data, formulating the learning problem as a regularized least squares problem where the regularizer favors smooth solutions within a trajectory. We develop a closed-form algorithm with guarantees of optimality and with a complexity that increases linearly with the number of instants considered per trajectory. The COSMIC algorithm achieves the desired result even in the presence of large volumes of data. Our method solved the problem using two orders of magnitude less computational power than a general purpose convex solver and was about 3 times faster than a Stochastic Block Coordinate Descent especially designed method. Computational times of our method remained in the order of magnitude of the second even for 10k and 100k time instants, where the general purpose solver crashed. To prove its applicability to real world systems, we test with spring-mass-damper system and use the estimated model to find the optimal control path. Our algorithm was applied to both a Low Fidelity and Functional Engineering Simulators for the Comet Interceptor mission, that requires precise pointing of the on-board cameras in a fast dynamics environment. Thus, this paper provides a fast alternative to classical system identification techniques for linear time-variant systems, while proving to be a solid base for applications in the Space industry and a step forward to the incorporation of algorithms that leverage data in such a safety-critical environment.

Predicting if passengers in a connecting flight will lose their connection is paramount for airline profitability. We present novel machine learning-based decision support models for the different stages of connection flight management, namely for strategic, pre-tactical, tactical and post-operations. We predict missed flight connections in an airline's hub airport using historical data on flights and passengers, and analyse the factors that contribute additively to the predicted outcome for each decision horizon. Our data is high-dimensional, heterogeneous, imbalanced and noisy, and does not inform about passenger arrival/departure transit time. We employ probabilistic encoding of categorical classes, data balancing with Gaussian Mixture Models, and boosting. For all planning horizons, our models attain an AUC of the ROC higher than 0.93. SHAP value explanations of our models indicate that scheduled/perceived connection times contribute the most to the prediction, followed by passenger age and whether border controls are required., Comment: Submitted to IEEE Transactions on Intelligent Transportation Systems

Orbit determination of spacecraft in orbit has been mostly dependent on either GNSS satellite signals or ground station telemetry. Both methods present their limitations, however: GNSS signals can only be used effectively in earth orbit, and ground-based orbit determination presents an inherent latency that increases with the Earth-spacecraft distance. For spacecraft flying formations, an alternative method of orbit determination, independent of external signals, consists in the observation of the spacecraft's position with respect to the central body through the relative positioning history of the spacecraft within the formation. In this paper, the potential of the relative positioning method is demonstrated in the context of the SunRISE mission, and compared with the mission's previously proposed orbit determination methods. An optimization study is then made to find the optimal placement of a new spacecraft in the formation so as to maximize the positioning accuracy of the system. Finally, the possibility of removing part of the system's relative bearing measurements while maintaing its observability is also studied. The resulting system is found to be observable, but ill-conditioned., Comment: 34 pages, 15 figures, submitted to Acta Astronautica

Space debris is a major problem in space exploration. International bodies continuously monitor a large database of orbiting objects and emit warnings in the form of conjunction data messages. An important question for satellite operators is to estimate when fresh information will arrive so that they can react timely but sparingly with satellite maneuvers. We propose a statistical learning model of the message arrival process, allowing us to answer two important questions: (1) Will there be any new message in the next specified time interval? (2) When exactly and with what uncertainty will the next message arrive? The average prediction error for question (2) of our Bayesian Poisson process model is smaller than the baseline in more than 4 hours in a test set of 50k close encounter events., Comment: Presented at AI4Spacecraft (IJCAI 2021 workshop)

As access to space and robotic autonomy capabilities move forward, there is simultaneously a growing interest in deploying large, complex space structures to provide new on-orbit capabilities. New space-borne observatories, large orbital outposts, and even futuristic on-orbit manufacturing will be enabled by robotic assembly of space structures using techniques like on-orbit additive manufacturing which can provide flexibility in constructing and even repairing complex hardware. However, the dynamics underlying the robotic assembler during manipulation may operate under inertial uncertainties. Thus, inertial estimation of the robot and the manipulated component system must be considered during structural assembly. The contribution of this work is to address both the motion planning and control for robotic assembly with consideration of the inertial estimation of the combined free-flying robotic assembler and additively manufactured component system. Specifically, the Linear Quadratic Regulator Rapidly-Exploring Randomized Trees (LQR-RRT*) and dynamically feasible path smoothing are used to obtain obstacle-free trajectories for the system. Further, model learning is incorporated explicitly into the planning stages via approximation of the continuous system and accompanying reward of performing safe, objective-oriented motion. Remaining uncertainty can then be dealt with using robust tube model predictive control. By obtaining controlled trajectories that consider both obstacle avoidance and learning of the inertial properties of the free-flyer and manipulated component system, the free-flyer rapidly considers and plans the construction of space structures with enhanced system knowledge. The approach naturally generalizes to repairing, refueling, and re-provisioning space structure components while providing optimal collision-free trajectories under e.g., inertial uncertainty.

Collaborative space robots are an emerging technology with high impact as robots facilitate servicing functions in collaboration with astronauts with higher precision during lengthy tasks, under tight operational schedules, with less risk and costs, making them more efficient and economically more viable. However, human-robot collaboration in space is still a challenge concerning key issues in human-robot interaction, including mobility and collaborative manipulation of objects on a microgravity environment. In this paper we formulate an algorithm that enables a free-flyer robot, equipped with a manipulator, to perform an object handover between a human and a free-flyer robot, in a microgravity environment. To validate and evaluate this algorithm, we present a systematic user study with the goal of understanding the subjective outcome effects of a rigid and compliant impedance robot behavior during the interaction. The results showed that the rigid behavior was overall more preferable and registered higher transfer success during the tasks., Comment: 8 pages, 4 figures, accepted for i-SAIRAS 2020

In order to ensure the robust actuation of a plan, execution must be adaptable to unexpected situations in the world and to exogenous events. This is critical in domains in which committing to a wrong ordering of actions can cause the plan failure, even when all the actions succeed. We propose an approach to the execution of a task plan that permits some adaptability to unexpected observations of the state while maintaining the validity of the plan through online reasoning. Our approach computes an adaptable, partially-ordered plan from a given totally-ordered plan. The partially-ordered plan is adaptable in that it can exploit beneficial differences between the world and what was expected. The approach is general in that it can be used with any task planner that produces either a totally or a partially-ordered plan. We propose a plan execution algorithm that computes online the complete set of valid totally-ordered plans described by an adaptable partially-ordered plan together with the probability of success for each of them. This set is then used to choose the next action to execute., Comment: Preprint, not submitted to any conference yet

Time perception is the phenomenological experience of time by an individual. In this paper, we study how to replicate neural mechanisms involved in time perception, allowing robots to take a step towards temporal cognition. Our framework follows a twofold biologically inspired approach. The first step consists of estimating the passage of time from sensor measurements, since environmental stimuli influence the perception of time. Sensor data is modeled as Gaussian processes that represent the second-order statistics of the natural environment. The estimated elapsed time between two events is computed from the maximum likelihood estimate of the joint distribution of the data collected between them. Moreover, exactly how time is encoded in the brain remains unknown, but there is strong evidence of the involvement of dopaminergic neurons in timing mechanisms. Since their phasic activity has a similar behavior to the reward prediction error of temporal-difference learning models, the latter are used to replicate this behavior. The second step of this approach consists therefore of applying the agent's estimate of the elapsed time in a reinforcement learning problem, where a feature representation called Microstimuli is used. We validate our framework by applying it to an experiment that was originally conducted with mice, and conclude that a robot using this framework is able to reproduce the timing mechanisms of the animal's brain., Comment: 13 pages, 6 figures

Robotic systems often operate with uncertainties in their dynamics, for example, unknown inertial properties. Broadly, there are two approaches for controlling uncertain systems: design robust controllers in spite of uncertainty, or characterize a system before attempting to control it. This paper proposes a middle-ground approach, making trajectory progress while also accounting for gaining information about the system. More specifically, it combines excitation trajectories which are usually intended to optimize information gain for an estimator, with goal-driven trajectory optimization metrics. For this purpose, a measure of information gain is incorporated (using the Fisher Information Matrix) in a real-time planning framework to produce trajectories favorable for estimation. At the same time, the planner receives stable parameter updates from the estimator, enhancing the system model. An implementation of this learn-as-you-go approach utilizing an Unscented Kalman Filter (UKF) and Nonlinear Model Predictive Controller (NMPC) is demonstrated in simulation. Results for cases with and without information gain and online parameter updates in the system model are presented.

The complex tasks such as surveillance, construction, search and rescue can benefit of the maneuverability of multirotor Micro Aerial Vehicles (MAVs) to obtain robust, cooperative system behavior and formation control is a prominent component of the these complex tasks. This work focuses on the problem of three-dimensional formation control of multirotor MAVs by using exclusively relative sensory information. It proposes a centralized Nonlinear Model Predictive Control (NMPC) approach in a leader-follower scheme. A realistic six degrees of freedom mathematical model of a multirotor MAVs is introduced and leveraged in the control laws. The formulation of the problem is performed based on NMPC and relative sensing framework with respect to local coordinate frames of the robots. This type of formulation makes the formation independent of the full knowledge of global or common reference frames and the utilization of expensive global localization sensors. Real-time Iteration (RTI) based solution to optimal control problem (OCP) is proposed by taking the novel formulation into account. An extensive scenario is designed to test and validate the strategy. Evaluation of the results suggests that satisfactory robust performance is achieved and maintained under model uncertainty and noise in local sensors and even in cases where the dynamics of the formation suddenly changes., Comment: 8 pages, 10 figures, IROS'2019 (submitted)

Using natural language to give instructions to robots is challenging, since natural language understanding is still largely an open problem. In this paper we address this problem by restricting our attention to commands modeled as one action, plus arguments (also known as slots). For action detection (also called intent detection) and slot filling various architectures of Recurrent Neural Networks and Long Short Term Memory (LSTM) networks were evaluated, having LSTMs achieved a superior accuracy. As the action requested may not fall within the robots capabilities, a Support Vector Machine(SVM) is used to determine whether it is or not. For the input of the neural networks, several word embedding algorithms were compared. Finally, to implement the system in a robot, a ROS package is created using a SMACH state machine. The proposed system is then evaluated both using well-known datasets and benchmarks in the context of domestic service robots.

The estimation of inertial parameters of a robotic system is crucial for better trajectory tracking performance, specially when model-based controllers are used for carrying out precise tasks. In this paper, we consider the scenario of grasping an object of unknown properties by a free-flyer space robot with limited actuation. The problem is to find the inertial parameters of the complete system after grasping has been performed. Excitation is provided in inertial space, and the excitation trajectories are found by optimization. Truncated Fourier series are used to represent the reference as well as tracked trajectory. An approach based on the energy balance between the actuation work and the rate of change of kinetic energy is introduced to calculate the number of harmonics in the Fourier series used to represent the executed trajectory, while trying to find a balance between accounting for saturation effects and keeping out noise. The effect of input saturation on parameter estimation is also studied. Simulation results using the Space CoBot free-flyer robot are presented to show the feasibility of the approach.

This paper presents the design of a small aerial robot for inhabited microgravity environments, such as orbiting space stations (e.g., ISS). In particular, we target a fleet of robots, called Space CoBots, for collaborative tasks with humans, such as telepresence and cooperative mobile manipulation. The design is modular, comprising an hexrotor based propulsion system, and a stack of modules including batteries, cameras for navigation, a screen for telepresence, a robotic arm, space for extension modules, and a pair of docking ports. These ports can be used for docking and for mechanically attaching two Space CoBots together. The kinematics is holonomic, and thus the translational and the rotational components can be fully decoupled. We employ a multi-criteria optimization approach to determine the best geometric configuration for maximum thrust and torque across all directions. We also tackle the problem of motion control: we use separate converging controllers for position and attitude control. Finally, we present simulation results using a realistic physics simulator. These experiments include a sensitivity evaluation to sensor noise and to unmodeled dynamics, namely a load transportation., Comment: Submitted to IROS 2016. Submission was moved to an update of a previous article. Now, it can be found at arXiv:1602.03573v2

This paper presents a first contribution to the design of a small aerial robot for inhabited microgravity environments, such as orbiting space stations. In particular, we target a fleet of robots for collaborative tasks with humans, such as telepresence and cooperative mobile manipulation. We explore a propeller based propulsion system, arranged in such a way that the translational and the rotational components can be decoupled, resulting in an holonomic hexarotor. Since propellers have limited thrust, we employ an optimization approach to select the geometric configuration given a criteria of uniform maximum thrust across all directions in the body reference frame. We also tackle the problem of motion control: due to the decoupling of translational and rotational modes we use separate converging controllers for each one of these modes. In addition, we present preliminary simulation results in a realistic simulator, in closed loop with the proposed controller, thus providing a first validation of the followed methodology., Comment: 21 pages, 13 figures. Previously at arXiv:1603.07545

The synchronization of dynamical systems is a method that allows two systems to have identical state trajectories, appart from an error converging to zero. This method consists in an appropriate unidirectional coupling from one system (drive) to the other (response). This requires that the response system shares the same dynamical model with the drive. For the cases where the drive is unknown, Chen proposed in 2002 a method to adapt the response system such that synchronization is achieved, provided that (1) the response dynamical model is linear with a vector of parameters, and (2) there is a parameter vector that makes both system dynamics identical. However, this method has two limitations: first, it does not scale well for complex parametric models (e.g., if the number of parameters is greater than the state dimension), and second, the model parameters are not guaranteed to converge, namely as the synchronization error approaches zero. This paper presents an adaptation law addressing these two limitations. Stability and convergence proofs, using Lyapunov's second method, support the proposed adaptation law. Finally, numerical simulations illustrate the advantages of the proposed method, namely showing cases where the Chen's method fail, while the proposed one does not., Comment: 14 pages, 7 figures

Most cognitive architectures rely on discrete representation, both in space (e.g., objects) and in time (e.g., events). However, a robot interaction with the world is inherently continuous, both in space and in time. The segmentation of the stream of perceptual inputs a robot receives into discrete and meaningful events poses as a challenge in bridging the gap between internal cognitive representations, and the external world. Event Segmentation Theory, recently proposed in the context of cognitive systems research, sustains that humans segment time into events based on matching perceptual input with predictions. In this work we propose a framework for online event segmentation, targeting robots endowed with active perception. Moreover, sensory processing systems have an intrinsic latency, resulting from many factors such as sampling rate, and computational processing, and which is seldom accounted for. This framework is founded on the theory of dynamical systems synchronization, where the system considered includes both the robot and the world coupled (strong anticipation). An adaption rule is used to perform simultaneous system identification and synchronization, and anticipating synchronization is employed to predict the short-term system evolution. This prediction allows for an appropriate control of the robot actuation. Event boundaries are detected once synchronization is lost (sudden increase of the prediction error). An experimental proof of concept of the proposed framework is presented, together with some preliminary results corroborating the approach., Comment: Technical report, 20 pages, 7 figures